COOPERATIVE CONTROL OF MULTI-AGENT SYSTEMS STABILITY, OPTIMALITY AND ROBUSTNESS KRISTIAN HENGSTER MOVRIĆ

Transcription

1 COOPERAIVE COROL OF MULI-AGE SYSEMS SABILIY, OPIMALIY AD ROBUSESS by KRISIA HEGSER MOVRIĆ Presented to the Faculty of the Graduate School of he Unversty of exas at Arlngton n Partal Fulfllment of the Requrements for the Degree of DOCOR OF PHILOSOPHY HE UIVERSIY OF EXAS A ARLIGO August 3

2 Copyrght by Krstan Hengster Movrc 3 All Rghts Reserved

3 ACKOWLEDGEMES I would lke to thank everyone who, n one way or another, provded me wth nvaluable support, gudance, counsel, comfort and love, durng these past years at the Unversty of exas at Arlngton. he tme spent here, workng towards ths degree, has n many ways been a lfe fulfllng and shapng experence. At the begnnng I could hardly foresee the scope of changes ths experence was to brng. I can only hope the changes are for the better. Frst of all, I express my everlastng grattude to my parents, Karmen and omca, who have contnuously encouraged me to follow the path I have chosen, and supported the choces I made along the way. Wthout your selfless, lovng dedcaton and support t would had been a much more dffcult and loneler path to traverse. You have been there, besdes me, every step of the way. I am grateful to all my teachers. In partcular to dr. Frank L. Lews, who thought me how to mantan research rgor and address problems n a most drect and effcent way, and to dr. Barbara A. Shpman, who thought me the value of precse, austere, logc and the great mportance of a defnton. I shall strve to keep your lessons and advce alve through the work that follows. I wsh to thank Kasa Sturlan, for makng me see n a way dfferent than before, for makng me realze so much of that, whch, otherwse, would have remaned gnored, mstaken for rrelevant, dormant, and slenced. hank you for compellng me to be. Your gft s chershed, more than I can express, and much more than I could ever hope to recprocate. Wthout all of you, I would not be n a poston to consder myself so fortunate. Words do not suffce to express my full grattude. Also, ths work s supported by SF grant ECCS-85, AFOSR EOARD Grant #3-355, Chna SF grant 66, and Chna Educaton Mnstry Project (o.b85). July 3, 3

4 ABSRAC COOPERAIVE COROL OF MULI-AGE SYSEMS; SABILIY, OPIMALIY AD ROBUSESS Krstan Hengster Movrć, PhD he Unversty of exas at Arlngton, 3 Supervsng Professor: Frank L. Lews In ths work desgn methods are gven for dstrbuted synchronzaton control of multagent systems on drected communcaton graphs. Condtons are derved based on the relaton of the graph egenvalues to a regon n a complex plane that depends on the sngle-agent system and the soluton of the local Rccat equaton. he synchronzng regon concept s used. Cooperatve observer desgn, guaranteeng convergence of the local estmates to ther true values, s also proposed. he noton of convergence regon for dstrbuted observers s ntroduced. A dualty prncple s shown to hold for dstrbuted observers and controllers on balanced graph topologes. Applcaton of cooperatve observers s made to the dstrbuted synchronzaton problem. hree dynamc regulator archtectures are proposed for cooperatve synchronzaton. In the second part ths work brngs together stablty and optmalty theory to desgn dstrbuted cooperatve control protocols, whch guarantee consensus and are globally optmal wth respect to a structured performance crteron. Here an nverse optmalty approach s used together wth partal stablty to consder cooperatve consensus and synchronzaton algorthms. v

5 A new class of dgraphs s defned admttng a dstrbuted soluton to the global optmal control problem. he thrd part of ths work nvestgates cooperatve control performance under dsturbances, and dstrbuted statc output-feedback control. Control desgn for the state consensus n presence of dsturbances s nvestgated. Derved results are also applcable to mult-agent systems wth heterogeneous agents. If, on the other hand, one constrans the control to be of the statc output-feedback form, one needs to redefne the synchronzng regon as the output-feedback synchronzng regon. Contrbutons to Dscrete-tme Mult-agent Consensus Problem he man contrbuton to the dscrete-tme mult-agent consensus problem s the proposed desgn method based on local Rccat feedback gans, guaranteeng cooperatve stablty and convergence to consensus. Contrbutons to Globally Optmal Dstrbuted Control Problem he globally optmal dstrbuted synchronzaton control protocols are nvestgated. he man contrbuton s n mergng the notons of nverse optmalty and partal stablty to guarantee robust stablzaton to the noncompact consensus manfold. Furthermore, second contrbuton s the ntroducton of the class of dgraphs that gves a dstrbuted soluton to a structured global optmal control problem. Contrbutons to Cooperatve Robustness of Mult-agent Systems he robustness propertes of asymptotc and exponental stablty are appled n the context of cooperatve stablty for consensus. he results are based on Lyapunov functons for noncompact manfolds, and the pertnent stablty and robustness propertes are further elaborated. Dstrbuted and local observers are utlzed for dsturbance compensaton. v

6 Contrbutons to Dstrbuted Output-feedback for State Synchronzaton An applcaton of the cooperatve stablty analyss, va synchronzng regon, to the dstrbuted output-feedback s presented. It s shown that the guaranteed synchronzng regon for output-feedback can be both bounded and unbounded. v

7 ABLE OF COES ACKOWLEDGEMES..... ABSRAC... v LIS OF ILLUSRAIOS...x Chapter Page. IRODUCIO Mult-Agent Systems and Synchronzaton..... Dstrbuted Control Dscrete-tme Cooperatve Control Desgn for Synchronzaton Dstrbuted Observer Desgn n Dscrete-tme Optmal Cooperatve Control Mult-agent Systems wth Dsturbances Output Feedback-Stablty and Optmalty Outlne of the Dssertaton DISCREE IME RICCAI DESIG OF LOCAL FEEDBACK GAIS FOR SYCHROIZAIO.... Introducton.... Graph Propertes and otaton....3 State Feedback for Synchronzaton of Mult-Agent Systems Desgn Based on H -type Rccat Inequalty Desgn based on H -type Rccat Equaton Relaton Between the wo Desgn Methods Robustness of the H Rccat Desgn Applcaton to Real Graph Matrx Egenvalues and Sngle-nput Systems umercal Example Concluson v

8 3. DISRIBUED OBSERVAIO AD DYAMIC OUPU COOPERAIVE FEEDBACK Introducton Cooperatve Observer Desgn Observer Dynamcs Desgn of Observer Gans Dualty Between Cooperatve Observer and Controller Desgn hree Regulator Confguratons of Observer and Controller Dstrbuted Observers and Controllers Local Observers and Dstrbuted Controllers Dstrbuted Observers and Local Controllers umercal Examples Example Example Concluson OPIMAL DISRIBUED COROL Introducton Asymptotc Stablty of the Consensus Manfold Inverse Optmal Control Optmalty Inverse Optmalty Optmal Cooperatve Control for Quadratc Performance Index and Sngle-ntegrator Agent Dynamcs Optmal Cooperatve Regulator Optmal Cooperatve racker Optmal Cooperatve Control for Quadratc Performance Index and Lnear me-nvarant Agent Dynamcs... 8 v

9 4.5. Optmal Cooperatve Regulator Optmal Cooperatve racker Constrants on Graph opology Undrected Graphs Detaled Balanced Graphs Drected Graphs wth Smple Laplacan Concluson OPIMAL COOPERAIVE COROL FOR GEERAL DIGRAPHS: PERFORMACE IDEX WIH CROSS-WEIGHIG ERMS Introducton Optmal Cooperatve Regulator- Sngle Integrator Agent Dynamcs Optmal Cooperatve racker- Sngle Integrator Agent Dynamcs Condton for Exstence of Dstrbuted Optmal Control wth Cross-weghtng erms n the Performance Index General Lnear me-nvarant Systems-Cooperatve Regulator General Lnear me-nvarant Systems-Cooperatve racker Concluson MULI-AGE SYSEMS WIH DISURBACES Introducton Stablty Defntons and Propertes Assessng the Effects of Dsturbances on Mult-agent Systems Usng Lyapunov Functons Mult-agent System Dynamcs wth Dsturbances Specfcaton to Lnear me-nvarant Systems Dsturbance Estmators Estmatng Dsturbances on the Agents Case of Identcal Dsturbance Generators for all Agents... 3 x

10 6.4.3 Estmatng Dsturbance on the Leader wth Known Leader's Dsturbance Generator Estmatng Dsturbance on the Leader Wthout the Leader's Dsturbance Generator Synchronzaton to the Leader Drven by an Unknow Input System Dynamcs Estmatng the Leader's Input Applcaton to Second-order Double-ntegrator Systems wth Dsturbances Applcaton of Dsturbance Compensaton to Dynamcal Dfference Compensaton Example of State Dependent Dsturbances- Agents' Dynamcs Satsfyng the Unform Growth Bound umercal Example Concluson COOPERAIVE OUPU FEEDBACK FOR SAE SYCHROIZAO Introducton Synchronzaton wth Dstrbuted Output Feedback Control Local Output Feedback Desgn Output Feedback Gan he Guaranteed Synchronzng Regon for Output Feedback H Synchronzaton of Mult-agent Systems on Graphs Case of Optmal Output Feedback Condtons on Graph opology Undrected Graphs Drected Graphs wth Smple Graph Matrx Concluson FUURE WORK... 8 x

11 8. Introducton Mult-agent System Synchronzaton Wth Control Delays Dynamcs of the Mult-agent System wth Constant, Unform, Input Sgnal Delay Stablty Results for Lnear Sngle-delay Retarded Functonal Dfferental Equaton Applcaton to the Complex Lnear Sngle-delay Retarded Functonal Dfferental Equaton Output Consensus of Heterogeneous Agents Mult-agent System Dynamcs and the Control Goal Dstrbuted Dynamc Control for Output Syncronzaton... REFERECES... 8 BIOGRAPHICAL IFORMAIO... 5 x

12 LIS OF ILLUSRAIOS Fgure Page. Motvaton for proof of heorem.. Rccat desgn crcle C(, r ) and coverng crcle C( c, r ) of graph matrx egenvalues (a) on-weghed graph egenvalues and ther coverng crcle. (b) Weghted graph Matrx egenvalues, ther coverng crcle, and the synchronzaton regon (dashed crcle) 37.3 State trajectores of 5 agents; (a) Frst, (b) Second and (c) hrd state component, for an mproper choce of c State space trajectores of 5 agents; (a) Frst, (b) Second and (c) hrd state component, for proper value of c (a) on-weghted graph egenvalues and ther coverng crcle. (b) Weghted graph matrx egenvalues, ther coverng crcle, and convergence regon of observer (dashed crcle) States of the nodes and the control node wth perfect state measurements (a) Frst state components, (b) Second state components, (c) hrd state components Frst state trackng error component wth perfect state measurements Dstrbuted observers and controllers. (a) Frst state components, showng synchronzaton, (b) Frst trackng error components, (c) Frst observer error components Local observers and dstrbuted controllers. (a) Frst state components, showng synchronzaton, (b) Frst trackng error components, (c) Frst observer error components Dstrbuted observers and local controllers. (a) Frst state components, showng synchronzaton, (b) Frst trackng error components, (c) Frst observer error components ξ... ξ 6. Dsturbances actng on agents Agents' dsturbance local estmate errors Agents' dsturbance dstrbuted estmate errors Comparson of local and dstrbuted agents' dsturbance estmate errors Leader's nput dstrbuted estmates x

13 6.6 Agents' frst states, agents dsturbances do not act, there s no leader nput compensaton- synchronzaton to the leader s not acheved Agents' frst states, leader's nput s estmated and compensated, agent dsturbances do not act, synchronzaton to the leader s acheved Agents' frst states, leader's nput s compensated, but dsturbances actng on the agents are not compensated. he synchronzaton s not acheved Agents' frst states, leader's nput s compensated, agents' dsturbances are locally estmated and compensated. Synchronzaton s acheved Agents' frst states, leader's nput s compensated, agents' dsturbances are dstrbutvely estmated and compensated. Synchronzaton s acheved he connected and dsconnected hyperbolc regons of heorem x

14 CHAPER IRODUCIO hs chapter presents a bref ntroducton to mult-agent systems consdered n ths work and the control problems as they appear throughout ths dssertaton. Concepts of partal stablty appled to synchronzaton are ntroduced as well as nverse optmalty of cooperatve control protocols. he ssue of the robustness of cooperatve control systems and the effect that the dsturbances exert on a mult-agent system are ntroduced. Possbly utlzaton of statc output-feedback s also addressed. opcs and contrbutons of the followng chapters are brefly summarzed here, provdng a layout of the body of work expounded n ths dssertaton.. Mult-Agent Systems and Synchronzaton he man topc of ths dssertaton s cooperatve control of mult-agent systems for synchronzaton. In a mult-agent system each agent s a subsystem, granted wth some form of autonomy that justfes the decomposton of a total system nto ts agent subsystems. It s assumed that agents are dynamcal systems wth nputs whose states resde n a state space x X, and ther dynamcs s descrbed by dfferental or dfference equatons. he state space of the total system s naturally taken as the drect product of state spaces of ndvdual agents, Xtot = X X X, wth the total state beng x = ( x, x,, x ). In case of all agents havng the same state space, X, the total space s Xtot = X. State consensus s a dynamc property of a mult-agent system, wth states as elements of de facto the same state space, x X, where lm t d( x, xj ) =, (, j), where d : X X R s the dstance functon on the state space. A subset S X tot characterzed by x = x, (, j), s an embeddng of X nto X tot, and we call t the consensus subset, for f the j total state of the system s n that subset then the mult-agent system s n the state of consensus. If all the state spaces are manfolds, so s the total space and the consensus subset

15 becomes the consensus manfold. Va an embeddng the consensus manfold has all the topologcal propertes of an ndvdual agent state space, n partcular f X s noncompact so wll be the consensus manfold as well. In case of agents havng dfferent state spaces the state consensus s generally not defned, but one can defne output consensus f all agent outputs are elements of the same output space, Y. Such problems do not appear when one consders collectons of dentcal agents, where state and output spaces are the same for every agent, together wth ther dynamcs.. Dstrbuted Control Achevng asymptotc stablty of a consensus manfold s the control goal, and the control to acheve that am s assumed dstrbuted. If one were to consder centralzed control then the mult-agent system control problem would reduce to controllng a sngle system, albet a large one. However, the assumpton, we choose to adopt here, s that each agent's nput sgnal can depend only on states of some proper subset of the collecton of all agents. hs assumpton s made due to nherently dstrbuted nature of mult-agent systems, where one cannot expect the total state to be avalable for the purpose of controllng a sngle agent. Even f ths were possble one would be faced wth the dffculty of ncreasng complexty of a controller wth the ncreasng number of agents. hs would mean greater communcaton load and prce of the controller. If the controller of each agent s constraned to use only the states of those agents that are n some neghborhood of the respectve agent, then one has a dstrbuted control structure. hs makes dstrbuted control eo pso a constraned control problem. Such controller does not suffer from ncreasng complexty when ncreasng the number of agents, snce t depends only on the lmted number of agents, those that are n some local neghborhood. hs work focuses prmarly on dentcal agents, havng lnear tme-nvarant dynamcs, and ther consensus problem, though references are made to heterogeneous agents.

16 .3 Dscrete-tme Cooperatve Control Desgn for Synchronzaton he last two decades have wtnessed an ncreasng nterest n mult-agent network cooperatve systems, nspred by natural occurrence of flockng and formaton formng. hese systems are appled to formatons of spacecrafts, unmanned aeral vehcles, moble robots, dstrbuted sensor networks etc.. Early work wth networked cooperatve systems n contnuous and dscrete tme s presented n,3,4,5,6,7. hese papers generally referred to consensus wthout a leader. By addng a leader that pns to a group of other agents one can obtan synchronzaton to a command trajectory usng a vrtual leader 5, also named pnnng control 8,9. he graph propertes complcate the desgn of synchronzaton controllers due to the nterplay between the egenvalues of the graph Laplacan matrx and the requred stablzng gans. ecessary and suffcent condtons for synchronzaton are gven by the master stablty functon, and the related concept of the synchronzng regon, n 9,,. For contnuous-tme systems synchronzaton was guaranteed 9,,3 usng optmal state feedback derved from the contnuous tme Rccat equaton. It was shown that, usng Rccat desgn for the feedback gan of each node guarantees an unbounded rght-half plane regon n the s-plane. For dscrete-tme systems such general results are stll lackng, though 4 deals wth sngle-nput systems and undrected graph topology and 5 deals wth multvarable systems on dgraphs. hese were orgnally nspred by the earler work of 6,7, concernng optmal logarthmc quantzer densty for stablzng dscrete tme systems..4 Dstrbuted Observer Desgn n Dscrete-tme Results from cooperatve control desgn n dscrete-tme can be appled wth needed modfcatons to the problem of dstrbuted observaton. Output measurements are assumed and cooperatve observers are specally desgned for the mult-agent systems. Potental applcatons are dstrbuted observaton, sensor fuson, dynamc output regulators for synchronzaton, etc. For the needs of consensus and synchronzaton control we employ the cooperatve tracker, or pnnng control 7,3,6. he key dfference between systems n contnuous-tme,,9,, and 3

17 dscrete-tme, 5,6,8, s n the form of ther stablty regon. More precsely, n contnuous-tme the stablty regon, as the open left-half s-plane, s unbounded by defnton, so the synchronzng regon can also be made unbounded. On the other hand, the dscrete-tme stablty regon, as the nteror of the unt crcle n the z-plane, s nherently bounded and, therefore, so are the synchronzng regons. hs makes condtons for achevng dscrete-tme stablty more strct than the contnuous-tme counterparts..5 Optmal Cooperatve Control Optmal cooperatve control was recently consdered by many authors- 33,34,36,37,38,39, to name just a few. Optmalty of a control protocol gves rse to desrable characterstcs such as gan and phase margns, that guarantee robustness n presence of some types of dsturbances 4,4. he common dffculty, however, s that n the general case optmal control s not dstrbuted 34,36. Soluton of a global optmzaton problem generally requres centralzed,.e. global, nformaton. In order to have local control that s optmal n some sense t s possble e.g. to consder each agent optmzng ts own, local, performance ndex. hs s done for recedng horzon control n 33, mplctly n 3, and for dstrbuted games on graphs n 35, where the noton of optmalty s ash equlbrum. In 37 the LQR problem s phrased as a maxmzaton problem of LMI's under the constrant of the communcaton graph topology. hs s a constraned optmzaton takng nto account the local character of nteractons among agents. It s also possble to use a local observer to obtan the global nformaton needed for the soluton of the global optmal problem, as s done n 34. In the case of agents wth dentcal lnear tme-nvarant dynamcs, 38 presents a suboptmal desgn that s dstrbuted on the graph topology. Optmal control for mult-agent systems s complcated by the fact that the graph topology nterplays wth system dynamcs. he problems caused by the communcaton topology n the desgn of global optmal controllers wth dstrbuted nformaton can be approached usng the noton of nverse optmalty, 4. here, one chooses an optmalty 4

18 crteron related to the communcaton graph topology to obtan dstrbuted optmal control, as done for the sngle-ntegrator cooperatve regulator n 36. hs connecton between the graph topology and the structure of the performance crteron can allow for the dstrbuted optmal control. In the case that the agent ntegrator dynamcs contans topologcal nformaton, 39 shows that there s a performance crteron such that the orgnal dstrbuted control s optmal wth respect to t..6 Mult-agent Systems wth Dsturbances When dsturbances act on the mult-agent system the control law desgned for the undsturbed system cannot generally guarantee that the control goal shall be attaned. However, buldng on the classcal results on the exstence of Lyapunov functons for asymptotcally stable systems, and ther use n assessng the effect that dsturbances exert on those systems, 46, t s possble to extend such reasonng to partally stable systems, 4, n partcular those systems that reach consensus or synchronzaton. Wth Lyapunov functons for asymptotc stablty one s able to ascertan the effect of dsturbances on the mult-agent system, and to derve condtons on those dsturbances that allow for asymptotc stablty or unform ultmate boundedness along the target set. Cooperatve asymptotc stablty s n that sense robust to ths specfc class of dsturbances. Furthermore, wth the means to quantfy the effect of dsturbances one also gans the ablty to compensate t by an approprate control law. Robustness of the dstrbuted synchronzaton control for the nomnal,.e. undsturbed, system guarantees cooperatve asymptotc stablty of consensus, or cooperatve ultmate unform boundedness wth respect to consensus, n presence of certan dsturbances, 46. hs nherent property can be exploted n specal cases of heterogeneous and nonlnear agents. evertheless, n case of general dsturbances one needs to compensate for ther effects n order to retan the qualtatve behavor of the nomnal system. For that purpose dsturbance estmates are used. In Chapter 6, dsturbances are assumed to act on both the leader and the 5

19 followng agents. herefore both the leader's and the agents' dsturbances need to be estmated and compensated. Local and dstrbuted estmaton schemes are employed for that purpose..7 Output-feedback-Stablty and Optmalty In realstc applcatons one generally cannot use the full state of each agent for feedback control purposes. hs dffculty s usually overcome by usng some observer to estmate the full state from the system's nputs and outputs. hs way one obtans a dynamc output controller. However ths controller has ts dynamcs and s more complcated then the statc state controller. Bearng ths n mnd, t behooves one to nvestgate condtons under whch the statc output control suffces. Statc output control combnes the smplcty of statc full state controllers wth the avalablty of the system outputs. o addtonal state observaton s needed. It was shown, n 6, that passve systems are able to output synchronze usng statc output feedback under very mld assumptons. However, here, one s prmarly nterested n the state synchronzaton, and therefore one can apply the concept of the synchronzng regon to statc output feedback. hs lne of thought reduces the mult-agent cooperatve stablty of consensus usng output dstrbuted feedback to robust output stablzaton for a sngle agent. Furthermore, under certan condtons, one can parameterze the cooperatvely stablzng output dstrbuted controllers by a quadratc optmalty crteron, as was done for the statc state dstrbuted control n Chapter 4. Chapter 7 brngs results along those lnes n a more general settng of two-player zero-sum games. It s shown that, under addtonal condtons, the proposed dstrbuted output-feedback s a soluton of the specally structured two-player zerosum game..8 Outlne of the Dssertaton he dssertaton s organzed as follows. Chapter ntroduces the dscrete-tme multagent systems and synchronzaton problem. A short survey of graph theoretc results that are used throughout the dssertaton s gven n ths chapter. Dscrete-tme synchronzng regon s ntroduced and dscrete-tme Lyapunov method gves a part of the synchronzng regon. 6

20 Dscrete-tme Rccat equaton (DARE) s used wth ts guaranteed complex gan margn regon to yeld a suffcent condton on graph egenvalues for synchronzaton. he work of Keyou You and Lhua Xe uses a modfed Rccat nequalty,.e. H -type Rccat nequalty, whle the author uses H -type Rccat equaton. Both methods are presented for the sake of completeness. Chapter 3 deals wth dstrbuted observaton and output-feedback cooperatve control. Contrary to the case presented n Chapter,6 and 8, perfect nformaton on the state of the neghbourng systems s not presumed. Output measurements are assumed and cooperatve observers are specally desgned for the mult-agent systems. Potental applcatons are dstrbuted observaton, sensor fuson, dynamc output regulators for synchronzaton, etc. Condtons for cooperatve observer convergence and for synchronzaton of the mult-agent system are shown to be related by a dualty concept for dstrbuted systems on drected graphs. Suffcent condtons are derved that guarantee observer convergence as well as synchronzaton. hs dervaton s facltated by the concept of convergence regon for a dstrbuted observer, whch s analogous, and n a sense dual, to the synchronzaton regon defned for a dstrbuted synchronzaton controller. Furthermore, the proposed observer and controller feedback desgns have a robustness property lke the one orgnally presented n Chapter, 8, for controller desgn. Chapter 4 ntroduces globally optmal dstrbuted synchronzaton protocols. In ths chapter are consdered fxed topology drected graphs and lnear tme-nvarant agent dynamcs. Frst, theorems are provded for partal stablty and nverse optmalty of a form useful for applcatons to cooperatve control, where the synchronzaton manfold may be noncompact. In our frst contrbuton, usng these results, we solve the globally optmal cooperatve regulator and cooperatve tracker problems for both sngle-ntegrator agent dynamcs and also agents wth dentcal lnear tme-nvarant dynamcs. It s found that globally optmal lnear quadratc regulator (LQR) performance cannot be acheved usng dstrbuted lnear control protocols on 7

21 arbtrary dgraphs. A necessary and suffcent condton on the graph topology s gven for the exstence of dstrbuted lnear protocols that solve a global optmal LQR control problem. In our second contrbuton, we defne a new class of dgraphs, namely, those whose Laplacan matrx s smple, that s, has a dagonal Jordan form. On these graphs, and only on these graphs, does the globally optmal LQR problem have a dstrbuted lnear protocol soluton. If ths condton s satsfed, then dstrbuted lnear protocols exst that solve the global optmal LQR problem only f the performance ndces are of a certan form that captures the topology of the graph. hat s, the achevable optmal performance depends on the graph topology. Chapter 5, extends results of Chapter 4 to slghtly more general but stll quadratc performance ndces. he constrant on graph topology of Chapter 4 can be relaxed, and optmal cooperatve controllers developed for arbtrary dgraphs, contanng a spannng tree, by allowng state-control cross-weghtng terms n the performance crteron. hen, requrng that the performance crteron be postve (sem) defnte leads to condtons on the matrx P n V ( x) = x Px, or equvalently on the control Lyapunov functon, whch should be satsfed for the exstence of dstrbuted globally optmal controller. hs condton s mlder than the condtons n Chapter 4, where the performance ndex s taken wthout the state-control cross-weghtng term. Chapter 6 dscusses the effects of dsturbances on the mult-agent systems. hose effects are quantfed by a Lyapunov functon for stablty as presented n Chapter 4. Propertes of relevant types of stablty are further elaborated for the needs of ths chapter. In partcular, the class of coordnate transformatons preservng the stablty propertes s nvestgated and characterzed. It s shown that systems that are asymptotcally or exponentally stable wth respect to some manfold share a robustness property to a certan class of dsturbances. hs class s characterzed by growth bounds wth respect to the target manfold. Snce most of the dsturbances are no expected to pertan to ths specfc class one resorts to dsturbance estmaton and compensaton. Leader's and agents' dsturbances are estmated and 8

22 compensated usng local and dstrbuted observers. A number of specal applcatons were consdered; the case of there beng an nput drvng the leader, whch needs to be dstrbutvely observed by all the followng agents, the case of second-order double-ntegrator systems wth dsturbances actng on the leader and the agents, and the case of heterogeneous agents. Chapter 7 brngs an extenson of the cooperatve stablty analyss va synchronzng regon, used orgnally for full-state feedback, to the case of output-feedback. It examnes statc dstrbuted output-feedback control for state synchronzaton of dentcal lnear tme-nvarant agents. Statc output-feedback, guaranteeng the control goal of mult-agent system synchronzaton, s easy to mplement, wthout any need for state observaton. Cooperatve stablty of state synchronzaton s addressed usng results derved from the sngle agent robust stablty propertes of the local output-feedback gan. It s shown that the guaranteed synchronzng regon for output-feedback can be both bounded and unbound. he chosen dstrbuted output-feedback control s also a soluton of the specally structured two-player zerosum game problem, under approprate addtonal stpulatons. Condtons for that are more conservatve than those for smple cooperatve synchronzaton. In a specal case ths game theoretc framework reduces to the optmal output-feedback control. Condtons are found, under whch the dstrbuted output-feedback control s optmal wth respect to a quadratc performance crteron. hese mply state synchronzaton. Chapter 8 outlnes future work drectons. Current results are brefly mentoned. Applcaton of the desgn methods developed n ths dssertaton to mult-agent systems wth fxed control sgnal tme-delay s partally developed. Identcal agents are descrbed by lnear delay-dfferental equatons. hs brngs one even closer to realty, snce dstrbuted communcaton causes delays n control sgnals. Also, a related feld of output synchronzaton s addressed. Output synchronzaton s practcally mportant n that one usually requres synchronzaton of only some states, or just outputs of the system, whle other states need only reman bounded. 9

23 CHAPER DISCREE IME RICCAI DESIG OF LOCAL FEEDBACK GAIS FOR SYCHROIZAIO. Introducton he last two decades have wtnessed an ncreasng nterest n mult-agent network cooperatve systems, nspred by natural occurrence of flockng and formaton formng. hese systems are appled to formatons of spacecrafts, unmanned aeral vehcles, moble robots, dstrbuted sensor networks etc.,. Early work wth networked cooperatve systems n contnuous and dscrete tme s presented n,3,4,5,6,7. hese papers generally referred to consensus wthout a leader. By addng a leader that pns to a group of other agents one can obtan synchronzaton to a command trajectory usng a vrtual leader 5, also named pnnng control 8,9. ecessary and suffcent condtons for synchronzaton are gven by the master stablty functon, and the related concept of the synchronzng regon, n 9,,. For contnuous-tme systems synchronzaton was guaranteed 9,,3 usng optmal state feedback derved from the contnuous tme Rccat equaton. It was shown that, usng Rccat desgn for the feedback gan of each node guarantees an unbounded rght-half plane regon n the s-plane. For dscrete-tme systems such general results are stll lackng, though 4 deals wth snglenput systems and undrected graph topology and 5 deals wth multvarable systems on dgraphs. hese were orgnally nspred by the earler work of 6,7, concernng optmal logarthmc quantzer densty for stablzng dscrete tme systems. In ths chapter we are concerned wth synchronzaton for agents descrbed by lnear tme-nvarant dscrete-tme dynamcs. he nteracton graph s drected and assumed to contan a drected spannng tree. For the needs of consensus and synchronzaton to a leader or control node we employ pnnng control 5,8. he concept of synchronzng regon 9,, s nstrumental n analyzng the synchronzaton propertes of cooperatve control systems. he synchronzng regon s the regon n the complex plane wthn whch the graph Laplacan matrx

24 egenvalues must resde to guarantee synchronzaton. he crucal dfference between systems n contnuous tme and dscrete tme s the form of the stablty regon. For contnuous-tme systems the stablty regon s the left half s-plane, whch s unbounded by defnton, and a feedback matrx can be chosen, 9,3 such that the synchronzng regon for a matrx pencl s also unbounded. On the other hand the dscrete-tme stablty regon s the nteror of the unt crcle n the z-plane, whch s nherently bounded. herefore, the synchronzng regons are bounded as well. hs accounts for strcter synchronzablty condtons n dscrete-tme, such as those presented n 4,5. In the semnal paper, 8, s gven an algorthm based on H -type Rccat equaton for synchronzaton control of lnear dscrete-tme systems that have no poles outsde the unt crcle. he case of consensus wthout a leader s consdered. hs chapter extends results n 5 to provde condtons for achevng synchronzaton of dentcal dscrete-tme state space agents on a drected communcaton graph structure. It extends results n 8 to the case of unstable agent dynamcs. hs work consders synchronzaton to a leader dynamcs. he concept of dscrete tme synchronzng regon n the z-plane s used. he graph propertes complcate the desgn of synchronzaton controllers due to the nterplay between the egenvalues of the graph Laplacan matrx and the requred stablzng gans. wo approaches to testng for synchronzablty are gven whch decouple the graph propertes from the feedback desgn detals. Both gve methods for selectng the feedback gan matrx to yeld synchronzaton. he frst result, based on an H -type Rccat nequalty, gves a mlder condton for synchronzaton n terms of a crcle whose radus s generally dffcult to compute. he second result s n terms of a crcle whose radus s easly computed from an H -type Rccat equaton soluton, but gves a strcter condton. Both are shown to yeld known results n the case of sngle-nput systems on undrected graphs. Based on the gven desgns,

25 results are gven on convergence and robustness of the desgn. An example llustrates the usefulness and effectveness of the proposed desgn. Consder a graph (, ). Graph Propertes and otaton G V E wth a nonempty fnte set of vertces V = { v,, v } and a set of edges or arcs E V V. It s assumed that the graph s smple,.e. there are no repeated edges or self-loops ( v, v ) E,. General drected graphs (dgraphs) are consdered, and t s taken that nformaton propagates through the graph along drected arcs. Denote the connectvty matrx as E = [ e j ] wth e > f ( v, v ) E and e = otherwse. ote that j j dagonal elements satsfy e =. he set of neghbors of node v s denoted as = { v : ( v, v ) E },.e. the set of nodes wth arcs comng nto v. Defne the n-degree j j j matrx as the dagonal matrx D dag ( d ) = d wth j j d = e the (weghted) n-degree of node (.e. the -th row sum of E ). Defne the graph Laplacan matrx as L = D E, whch has all row sums equal to zero. A path from node v to node v s a sequence of edges k ( v, v ), ( v, v ), ( v, v ), wth ( v v ) E or (, ) 3 k k, j j v v E for j {,, k} j j =. A drected path s a sequence of edges ( v, v ),( v, v ),,( v, v ), wth ( v v ) {,, k} 3 k k, j j E for j =. he graph s sad to be connected f every two vertces can be joned by a path. A graph s sad to be strongly connected f every two vertces can be joned by a drected path. he graph s sad to contan a (drected) spannng tree f there exsts a vertex, v, such that every other vertex n V can be connected to v by a (drected) path startng from v. Such a specal vertex, v, s then called a root node.

26 he Laplacan matrx L has a smple zero egenvalue f and only f the undrected graph s connected. For drected graphs, the exstence of a drected spannng tree s necessary and suffcent for L to have a smple zero egenvalue. A bdrectonal graph s a graph satsfyng e > e >. A detaled balanced graph s a graph satsfyng λ e = λ e j j j for some postve constants λ... λ. By summng over the ndex t s seen that then [ λ λ ] s a left zero egenvector of L. he Laplacan matrx of a detaled balanced graph satsfes L matrx Λ > and Laplacan matrx symmetrzable. In fact Λ = dag(/ λ ) the reversblty of an assocated Markov process. Gven the graph = (, ) P j j = Λ P, for some dagonal = P of some undrected graph. hat s, the Laplacan s. he concept of a detaled balanced graph s related to G V E, the reversed graph G ' s a graph havng the same set of vertces V ' = V and the set of edges wth the property E ' = e ' j = E = e j. hat s, the edges of G are reversed n numbers by C. radus. G '. We denote the real numbers by R, the postve real numbers by + R, and the complex C ( O, r ) denotes an open crcle n the complex plane wth ts center at O C and r he correspondng closed crcle s denoted as (, ) For any matrx A, ( A), ( A) mn max C O r. σ σ, are the mnmal and the maxmal sngular values of A respectvely. For any square matrx A, the spectral radus ( A) max eg ( A) ρ = s the maxmal magntude of the egenvalues of A. For a postve semdefnte matrx A, 3

27 σ ( A) > mn denotes the mnmal nonzero sngular value. ote that for symmetrc L, σ ( L) λ ( L) = > mn, the graph Fedler egenvalue..3 State Feedback for Synchronzaton of Mult-Agent Systems Gven a graph (, ) n G V E, endow each of ts nodes wth a state vector x R and a control nput, m u R, and consder at each node the dscrete-tme dynamcs ( ) ( ) ( ) x k + = Ax k + Bu k. () Assume that ( A, B) s stablzable and B has full column rank m. Consder also a leader node,.e. control node or command generator wth n ( ) Ax ( k) x k + =, () x R. For example, f n = and A has magnary poles, then the leader's trajectory s a snusod. he cooperatve tracker or synchronzaton problem s to select the control sgnals u, usng the relatve state of node to ts neghbors, such that all nodes synchronze to the state of the control node, that s, ( ) ( ) lm x k x k =,. hese requrements should be fulflled for k all ntal condtons, x ( ). If the trajectory x ( ) the consensus problem. k approaches a fxed pont, ths s usually called o acheve synchronzaton, defne the local neghborhood trackng errors ( ) ( ) ε = e x x + g x x, (3) j j j where the pnnng gan, g, s nonzero f node v can sense the state of the control node. he ntent here s that only a small percentage of nodes have g >, yet all nodes should synchronze to the trajectory of the control node usng local neghbor control protocols, 8. It s 4

28 assumed that at least one pnnng gan s nonzero. ote that the local neghborhood trackng error represents the nformaton avalable to agent for control purposes. Choose the nput of agent as the weghted local control protocol ( ) u = c + d + g Kε, (4) + where c R s a couplng gan to be detaled later. hen, the closed-loop dynamcs of the ndvdual agents are gven by x ( k + ) = Ax ( k) + c( + d + g ) BKε ( k). (5) n Defnng the global trackng error and state vector ε = ε ε R, n x = x x R, one may wrte where G dag ( g g ) ( k) ( L G) I x( k) ( L G) I x ( k) ε = + + +, (6) n =,, s the dagonal matrx of pnnng gans and x ( k) = x ( k) wth R the vector of s. he global dynamcs of the -agent system s gven by ( + ) = ( + + ) ( + ) ( ) + ( + + ) ( + ) ( ) x k I A c I D G L G BK x k c I D G L G BKx k. (7) Defne the global dsagreement error, δ ( k) = x( k) x ( k), 3. hen one has the global error dynamcs where the closed-loop system matrx s We shall refer to the matrx c δ ( k ) A δ ( k) c n + =, (8) ( ) ( ) A = I A c I + D+ G L+ G BK. (9) Γ = ( I + D + G) ( L + G) () 5

29 as the (weghted) graph matrx and to ts egenvalues, Λ, k =..., as the graph matrx k egenvalues. Assume the graph contans a drected spannng tree and has at least one nonzero pnnng gan, g, connectng nto the root node. he graph matrx Γ s nonsngular snce L + G s nonsngular, f at least one nonzero pnnng gan, g, connects nto the root node, 9. he followng result shows the mportance of usng weghtng by ( I D G) + +. Lemma.. Gven the control protocol (4), the egenvalues of Γ satsfy C (,) Λ, k k =..., for any graph. Proof: hs follows drectly from the Geršgorn crcle crteron appled to ( I D G) ( L G) d + g d whch has Geršgorn crcles C, + d + g + d + g Γ= + + +,. hese are all contaned n (,) C. Lemma. reveals that weghtng restrcts the possble postons of the graph egenvalues to the bounded known regon (,) graph egenvalues n the regon C( d + g, d ) C. he non-weghted protocol u = ckε yelds, whch s larger than (,) C. Examples. and. show stuatons where synchronzaton usng control law of the form u = ckε cannot be guaranteed, whereas usng weghted protocol (4) acheves synchronzaton. he next result s smlar to results n 6, 9, 4. ρ for all Lemma.. he mult-agent systems (5) synchronze f and only f ( A cλ BK ) < k egenvalues Λ, k =..., of graph matrx (). k Proof: Let J be a Jordan form of Γ. hen there exsts a nonsngular matrx R R, such that R R Γ = J. By (9), t s obtaned that 6

30 ( ) ( ) n c n R I A R I = I A cj BK A cλ BK =, () A cλ BK where denotes possbly nonzero elements. By (), one has ρ ( ) ( A c BK ) ρ Λ < for all k k A c < f and only f Λ. From (8), the mult-agent systems (5) synchronze f and only f ρ ( A c ) <. hus, the mult-agent systems (5) synchronze f and only f ( A cλ BK ) < ρ for all k Λ k. For synchronzaton, one requres asymptotc stablty of the error dynamcs (8). It s assumed that ( A, B ) s stablzable. If the matrx A s unstable or margnally stable, then Lemma. requres that Λ k, k =..., whch s guaranteed f the nteracton graph contans a spannng tree wth at least one nonzero pnnng gan nto the root node. Defnton.: Let Γ be a graph matrx, and let Λ, k =..., be the egenvalues of Γ. A k coverng crcle, (, ) C c r, of the egenvalues of Γ s an open crcle centered at c R contanng all the egenvalues, Λ, k =,...,. k Defnton.: For a matrx pencl, A sbk wth s C, the synchronzng regon of the matrx pencl s a subset S C such that S s ρ ( A sbk ) c c { } = C <. Gven the choce of K, the synchronzng regon S of the matrx pencl A sbk s c determned. he synchronzng regon was dscussed n 9,,, 3. Defnton.3: he complex gan margn regon, U c C, for some stablzng feedback matrx A sbk s U c K, gven a system ( A, B), s a connected regon contanng, such that ρ( ) <,. 7

31 Defntons. and.3 express the connecton between the synchronzng regon S c of a matrx pencl A sbk and the complex gan margn regon U c, gven that the choce of K stablzes the system (, ) A B,.e. ( A BK ) < ρ. he complex gan margn regon, defned as a connected regon, s contaned n the synchronzng regon, whch may be generally dsconnected. In 8, an example of a dsconnected synchronzng regon was gven. Lemma.3. Matrces A cλ BK are stable for all egenvalues Λ k f and only f they satsfy k cλ S, k {,, }. k Proof: hs follows from Defnton.. Based on these constructons, the man results of ths chapter may now be stated. We develop two approaches that provde suffcent condtons for synchronzaton and gve formal desgn methods for the synchronzng control (4), one based on an H -type Rccat nequalty and one based on an H -type Rccat equalty..4 Desgn Based on H -type Rccat Inequalty In what follows, the desgn of K n (4) wll be studed based on a modfed H -type Rccat nequalty. Wthout loss of generalty, we further assume that A = dag( A, A ), where A s ns ns nu nu R s stable and all the egenvalues of A R le on or outsde the unt crcle. u s u Accordngly, wrte B = [ B B ]. s u Let us frst analyse the structure of the system. Snce ( A, B ) s stablzable, the unstable part ( A, B ) s controllable. From the mult-nput reachable canoncal form, 43, there u u exsts a non-sngular real matrx V such that and A = V AuV take the form B = V B u 8

32 A b A b A, B = =, A m b m () where denotes possbly nonzero elements and ( A, b ) s controllable. ote that elements of matrces n () need not be scalars, but rather blocks n general. Usng the above form, a result on the soluton to a modfed Rccat nequalty can be developed. n n n n Lemma.4. Assume that M R and M R are two symmetrc postve defnte n n matrces. Gven any M R, let j j σ ( M M M ) max β >. (3) σ ( M ) mn hen M M M M M s a postve defnte matrx. β Proof: Accordng to the Schur complement condton, a symmetrc block matrx M = M M M M s postve defnte f both M and M M M M are postve defnte. ow, t follows from (3) that β M M M M βσ ( M ) I σ ( M M M ) I >. mn max ogether wth M >, one has from Schur complement that M >. Lemma.5. Consder a sngle nput state-space system wth A beng unstable and ( A, B ) stablzable. hen, a necessary and suffcent condton for the exstence of a postve defnte matrx P > solvng > δ (4) P A PA ( ) A PB( B PB) B PA s 9

33 u δ < / λ ( A). Proof: (ecessty) In conformty wth the partton A = dag( A, A ), partton P as s u P P P s = P P u. Pre- and post-multplyng (4) by I P P u I and I, yelds Pu P I P A P A A P B B P B B P A > ( δ ) ( ), u u u u u u u u u u u u u where B = B + P P B By contnuty, there exsts a suffcently small ε > such that u u u s. δ ε = A ( P + B B / ε) A + δ A P B ( B P B + ε) B P A u u u u u u u u u u u u u u >. P > A P A ( ) A P B ( B P B + ) B P A u u u u u u u u u u u u u he equalty n the second lne follows by applyng the Kalath varant. Snce both sdes of the nequalty are postve defnte, takng the determnant of both sdes gves det( P ) > det( A ) det( P ) det( I ( δ ) P B ( B P B + ε ) B ) u u u u u u u u u B P B u u = ( δ ) det( A ) det( P ) u u B P B + ε u u u > δ det( A ) det( P ), u u where the equalty holds due to the dentty det( I M) = det( I M ) for compatble matrces M,. he fact that u det( Au ) / ( A). δ < s used n the last nequalty. ote that det( u ). P > mples δ < = λ Suffcency follows from Lemma 5.4 of. For mult-nput systems, one has the followng result. Results of Lemma.4 and Lemma.5 are used n the proof. Lemma.6. Gven δ (,], consder the modfed H -type Rccat nequalty

34 P A PA A PB B PB B PA ( ) ( > δ ). (5) Assume that A s unstable and ( A, B ) s stablzable. hen there exsts a crtcal value δ c (,) such that f δ < δ, then there exsts a postve defnte soluton c P to (5). Moreover, δ c s bounded below by δ / λ ( A ) δ, (6) u c * m c where u A s gven n (), ( A ) j λ j are the unstable egenvalues of j A, and the ndex * m s defned by m * u = arg max λ ( Aj ). Furthermore, f A s stable, one has that δ c =. j {,, m} Proof: Snce ( A, B ) s stablzable, there exsts a P > solvng the nequalty P > A PA A PB B PB B PA ( ). hen, a suffcently small δ > exsts such that P > A PA A PB B PB B PA ( ) ( δ ). In addton, for any postve ζ δ, one has that P > A PA ζ A PB B PB B PA ( ) ( ). Hence, the exstence of a postve δ c, as requred n the lemma, s guaranteed. In fact, t can be obtaned as { δ P s.t. δ ) ( B) P } δ = sup > > ( (7) P A PA A PB B P B A c δ > Let m be the number of nputs. a) If m =, whch corresponds to B beng a vector, t follows from Lemma.5 u that δ = / λ ( A) = δ. herefore, (6) holds. c c For the mult-nput case, the result for sngle-nput system,.e. Lemma.5, s used n steps together wth () and Lemma.4 to construct the followng result.

35 b) Assume that exsts a P j > such that m >. Case a) mples that gven any postve δ δ c <, there P A P A + ( δ ) A P b ( b P b ) b P A = ( )( ) ( ) >, j j j j j j j j j j j j j Pj δ A j Pj A j δ A j b j K j Pj A j b j K j (8) wth the control gan K = ( b P b ) b P A. ote that for any β >, P j = βp > also j j j j j j j j solves the modfed Rccat nequalty (8). ow, let P = d a g( P, βp ) and K = dag( K, K ). It follows that P δ A P A ( δ )( A B K ) P ( A B K ) M M M, M βm where M > and j M are ndependent of β and A b, B A b A = =. By Lemma.4, there exsts a suffcently large β such that M >. Contnung n the same fashon, one can fnd P and K such that P δ δ A PA ( )( A B K ) P ( A B K ) >. (9) Let Pu V P = V and Ku = KV. It follows from (9) that P δ A P A ( δ )( A B K ) P ( A B K ) >. u u u u u u u u u u u Snce A s s stable, there exsts a postve defnte matrx P s such that s s s s P A P A >. Denotng K = dag(, K u ) and P = dag( Ps, βpu ), one can smlarly establsh that there exsts a postve β such that δ ( δ )( ) ( ) >. () P A PA A BK P A BK

36 ote that for any K m n R, one can verfy that ( A BK) P( A BK) A PA A PB( B PB) B PA. Whence, together wth (), t follows that P > A PA A PB B PB B PA ( ) ( δ ). hus, gven any δ < δ c, there exsts a P > whch solves the modfed Rccat nequalty (5). hs mples that. δc δ c If A s stable, t follows that δ = snce for any postve δ δc c <, there always exsts a postve defnte soluton to (5). In general, δ can be found by solvng an LMI, see Proposton 3. of. c λ u he term appearng n Lemma.6 nvolvng a product of unstable egenvalues, ( A) u, deserves further attenton. It s related to the ntrnsc entropy rate of a system, ( ) log λ A, descrbng the mnmum data-rate n networked control system that enables stablzaton of an unstable system,. he product of unstable egenvalue magntudes of matrx A tself s the Mahler measure of the respectve characterstc polynomal of A. It s well recognzed that Lemma.6 s of mportance n the stablty analyss of Kalman flterng wth ntermttent observatons,, and the quadratc stablzaton of an uncertan lnear system (cf. heorem. of 7). In fact, t s also useful for the desgn of a control gan n (4) to solve the synchronzaton problem, whch s delvered as follows. heorem.. H -type Rccat Inequalty Desgn for Synchronzaton. Gven systems () wth protocol (4), assume that the nteracton graph contans a spannng tree wth at least one pnnng gan nonzero that connects nto a root node. hen Λ >,. If there exsts ω R such that δ ( ω) max ω δ, j {,, } Λ j < c () 3

37 where δ c s obtaned by (7), then there exsts P > solvng (5) wth δ = δ ( ω). Moreover, the control gan and couplng gan, c ( K B PB) B PA =, = ω, guarantee synchronzaton. Proof: By Lemma.6, there exsts P > solvng (5). hus, only the second part needs to be shown. Denote δ = ωλ, t s found that j j * P ( A ωλ BK ) P( A ωλ BK ) j j = P A PA + ( δ ) A PB( B PB) B PA j P A PA + ( δ ) A PB( B PB) B PA >, where * denotes complex conjugate transpose. hus, t follows that ρ( A ωλ BK ) <. he rest j of the result follows from Lemma.. Remark.. For the sngle nput case,.e., rank( B )=, the suffcent condton gven n heorem. s also necessary. hs follows drectly from heorems 3. and 3. of 4. In 8, an equaton smlar to (5) s used for synchronzaton desgn n the case where there s no leader, and no poles of the agent dynamcs are outsde the unt crcle. he condtons of Lemma. are awkward n that the graph propertes, as reflected by the egenvalues Λ, and the stablty of the local node systems, as reflected n ( A BK), are k coupled. heorem. s mportant because t shows that feedback desgn based on the modfed Rccat nequalty allows a separaton of the feedback desgn problem from the propertes of the graph, as long as t contans a spannng tree wth a nonzero pnnng gan nto the root node. Specfcally, t reveals that f Rccat-based desgn s used for the feedback K at each node, then synchronzaton s guaranteed for a class of communcaton graphs satsfyng condton (). hs condton s appealng because t allows for a dsentanglement of the 4

38 propertes of the ndvdual agents feedback gans, as reflected by δ c, and the graph topology, descrbed by Λ k, relatng these through an nequalty..5 Desgn Based on H -type Rccat Equaton An H -type Rccat desgn method for fndng gan K n (4) that guarantees synchronzaton s now presented. heorem.. H -type Rccat Desgn for Synchronzaton. Assume that the nteracton graph contans a spannng tree wth at least one pnnng gan nonzero that connects nto the root node. hen Λ >,. Let P > be a soluton of the dscrete-tme Rccat-lke equaton A PA P + Q A PB( B PB) B PA = () for some prescrbed Q = Q >. Defne / / / r : = σ ( Q A PB( B PB) B PAQ ) max. (3) hen the protocol (4) guarantees synchronzaton of mult-agent systems (5) for some K f there exsts a coverng crcle (, ) that C c r of the graph matrx egenvalues Λ k, k = such r r c <. (4) Moreover, f condton (4) s satsfed then the choce of feedback matrx, ( B PB) B PA K =, (5) and couplng gan, c =, (6) c guarantee synchronzaton. 5

39 heorem. s motvated by geometrcal consderatons as shown n Fgure.. Condton (4) means that the coverng crcle (, ) and contaned n the nteror of C (, ) regon of the C c r s homothetc to a crcle concentrc wth r. It wll be shown n Lemma.7 that the synchronzng H -type Rccat feedback gan selected as n heorem. contans (, ) herefore, synchronzaton s guaranteed by any value of c radally projectng (, ) nteror of C(, ) C r. C c r to the r. One such value s gven by (6). Homothety, as understood n ths work, refers to a radal projecton wth respect to the orgn,.e. zero n complex plane. r r c Fgure. Motvaton for proof of heorem.. Rccat desgn crcle C (, ) of graph matrx egenvalues. he followng techncal lemmas are needed for the proof of heorem.. 6 r and coverng crcle Lemma.7. he synchronzng regon for the choce of K gven n heorem. contans the open crcle C(, r ). Proof: By choosng the state feedback matrx as (5), wth P > a soluton of equaton () one has (, r ) C c ( ) A PA P Q K B PB K From ths equaton one obtans by completng the square + =. (7) ( ) ( ) A PA P Q K B PBK A BK P A BK P Q + = + =. (8)

40 herefore, gven the quadratc Lyapunov functon ( ) V y y Py =, n y R, the choce of feedback gan stablzes the system ( A, B). At least a part of the synchronzng regon S can be found as * ( ) ( ) ( Re ) ( Re ) Im ( ) A sbk P A sbk = A sbk P A sbk + s BK PBK < P, (9) where * denotes complex conjugate transpose (Hermtan adjont), s C. herefore from equatons (7) and (9) one has * ( ) ( ) A sbk P A sbk P = + + A PA P Re sk ( B PB) K Re s( BK) PBK Im s( BK) PBK = A PA P ( s )( BK) PBK = Q+ K ( B PB) K ( s )( BK) PBK (3) = Q+ s K B PBK hs s the condton of stablty f Q s K B PBK >, (3) whch gves a smple bounded complex regon, more precsely a part of the complex gan margn regon U, hs s an open crcle C (, ) ( ) / / σ max Q > s K B PBK > s Q ( BK) P( BK) Q. (3) r specfed by s < σ. (33) max / / ( Q K B PBKQ ) Furthermore expressng K as the H -type Rccat equaton state feedback completes the proof. 7

41 In 8, t s shown that n case of all the agent dynamcs beng margnally stable, that s, no unstable poles and only nonrepeated poles on the unt crcle, the synchronzng regon (n the sense of ths chapter) s C (,). Lemma.8. Gven the crcle C (, ) r n the complex plane, whch s contaned n the synchronzng regon S for the H -type Rccat choce of gan (5), the system s guaranteed to synchronze for some value of c > f the graph matrx egenvalues Λ k, k =..., are located n such a way that for that partcular c n (6). cλ < r, k, (34) k Proof: It follows from Lemmas.,.3, and.7. Based on these constructons the proof of heorem. can now be gven. Proof of heorem.. Gven C (, r ) contaned n the synchronzng regon S of matrx pencl A sbk, and the propertes of dlaton (homothety), and assumng there exsts a drected spannng tree n the graph wth a nonzero pnnng gan nto the root node, t follows that synchronzaton s guaranteed f all egenvalues Λ are contaned n a crcle (, ) k C c r smlar wth respect to homothety to a crcle concentrc wth and contaned wthn C (, r ). he center of the coverng crcle c can be taken on the real axs due to symmetry and the radus equals r = max Λ c k. akng these as gven, one should have k r r c <. (35) 8

42 If ths equaton s satsfed then choosng c = / c maps wth homothety the coverng crcle of all egenvalues C( c, r ) nto a crcle (, ) C r c concentrc wth and, for r > r / c, contaned n the nteror of the crcle C (, r ). If there exsts a soluton P > to Rccat equaton (), B must have full column rank. Assumng B has full column rank, there exsts a postve defnte soluton P to () only f ( A, B) s stablzable..6 Relaton Between the wo Desgn Methods In ths secton the relaton between heorem. and heorem. s drawn. By comparng (34) to () t s seen that a smlar role s played by δ c n heorem. and r n heorem.. he latter can be explctly computed usng (3), but t depends on the selected Q. On the other hand, δ c s found by numercal LMI technques. See Lemma.6. Further, n heorem., ω R plays a role analogous to / c n heorem.. heorem. reles on analyss based on the crcle C(, δ ) c, whereas heorem. uses C(, ) contaned n the synchronzng regon of the respectve desgned feedbacks. r. Both crcles are More mportantly, heorem. gves synchronzaton condtons n terms of the radus r easly computed as (3) n terms of an H -type Rccat equaton soluton. Computng the radus δ c used n heorem. must generally be done va an LMI. However, the condton n heorem. s mlder than that n heorem.. hat s, C(, r ) s generally contaned n C(, δ ) c, so that systems that fal to meet the condton of heorem. may yet be found to be synchronzable when tested accordng to the condton n heorem.. hs result s summarzed n the followng theorem. heorem.3. Defne r by (3) and δ c as n (7). hen 9 r δ. c

43 Proof: Assume there exsts a postve defnte soluton to (5) for some δ, denoted as P δ, then t follows that A Pδ A Pδ A Pδ B( B Pδ B) B Pδ A < A Pδ B( B Pδ B) B Pδ A δ hs means, Q Q > δ A P B B P B B P A such that δ ; δ δ ( δ ) δ A Pδ A Pδ A Pδ B( B Pδ B) B Pδ A Qδ =. herefore, there exsts a postve defnte soluton of the Rccat equaton () for such Q δ. Accordng to the exstence of postve defnte soluton to (5) for < < there also exsts δ δ c the soluton of () wth the correspondng Q δ. Snce the guaranteed synchronzaton regon for each P δ s s < δ choosng Q δ ; δ < δ c, although guaranteeng the exstence of postve defnte P δ, thus provdng suffcent condton for synchronzaton, does not necessarly gve the largest guaranteed crcular synchronzaton regon,.e. generally s < r δc. ote that followng the desgn proposed by heorem. the matrx Q s gven, and t explctly determnes the sze of the guaranteed synchronzng regon (3) f there exsts a postve defnte soluton of () for that gven choce of Q. Fndng general Q > that maxmzes r would nvolve an LMI just as fndng the value of δ c does..7 Robustness of the H Rccat Desgn he followng lemma s motvated by the conjecture that f the condtons of heorem. hold there s n fact an open nterval of admssble values of the couplng constant c. hs has the nterpretaton of robustness for the H -type Rccat desgn n sense that synchronzaton s stll guaranteed under small perturbatons of c from value gven by (6). 3

44 Lemma.9. akng α : = arccos r and denotng the angle of an egenvalue of the max graph matrx Γ by φ : = arg Λ, a necessary condton for the exstence of at least one admssble value of c s gven by k k Re k Λ k r, k, (36) whch s equvalent to φ < α, k. max (37) k Furthermore, the nterval of admssble values of c, f non-empty, s an open nterval gven as a soluton to a set of nequaltes cosφ cos φ cos α cosφ + cos φ cos α k k max k k max < c < Λ k Λ k, k. (38) Fnally f (4) holds then the soluton of (38) s non-empty and contans the value (6). Proof: If soluton c exsts, the equatons k k k cλ < r c Λ c Re Λ + r < (39) are satsfed smultaneously for every k for at least one value of c. herefore k (39) has an nterval of real solutons for c. From that, and the dscrmnant of (39), relaton (36) follows. One therefore has Re Λ k cos φ = > r = cos k Λ k α max meanng φk < α. Bearng that n mnd, max expresson (36) can further be equvalently expressed as k k k c Λ c Λ cosφ + cos α <, max Solvng ths equaton for c Λ k yelds ntervals n (38). he ntersecton of these open ntervals s ether an open nterval or an empty set. ow gven that C( c, r ) that 3 Λ one fnds k

45 cosφk cos φk cos αmax cosφk cos φk cos α < Λ Λ k k max k mn cosφk + cos φk cos αmax cosφk + cos φk cos α < Λ Λ k max max (4) where Λ, Λ are extremal values for fxed k k mn k max φ, as determned by C ( c, r ). amely, for Λ, Λ one has k mn k max Λk Λk r r φ k c c c Λ c = cos + = k whch gves r Λ = c cos φ ± cos φ k max,mn k k c. If (4) s satsfed then, cos α < r max, c used n (4) together wth expresson for Λ, Λ mples the non-emptyness of the nterval k mn k max soluton of (38), and guarantees that c = c s a member of that nterval. Furthermore, f the assumptons of heorem. are satsfed wth equalty, then the lower and hgher lmt of every subnterval (4) n fact become equal to / c. Condton (37) means that the graph matrx egenvalues Λ k must be nsde the cone n complex plane shown n Fgure.. It s evdent from the geometry of the problem that egenvalues regon C(, ) Λ k located outsde the cone determned by (36), (37) cannot be made to ft nto the r by scalng wth real values of c. It should be noted that condton (4) s stronger than condton (37) and condton (38) has also been derved n 4..8 Applcaton to Real Graph Matrx Egenvalues and Sngle-nput Systems In ths secton condton (4) of heorem. s studed n several specal smplfed cases and t s shown how ths condton relates to known results. he specal case of real 3

46 egenvalues of Γ and sngle nput systems at each node allows one to obtan necessary and suffcent condton for synchronzaton. Corollary.. Let graph G ( V, E ) have a spannng tree wth pnnng nto the root node, and all egenvalues of Γ be postve, that s, Λ k > for all k. Defne <Λmn Λk Λmax. A coverng crcle for Λ k that also mnmzes / r c, s (, ) C c r wth r c Λ Λ =. (4) Λ + Λ max mn max mn hen, condton (4) n heorem. becomes Λ Λ max max Λ + Λ mn mn < r. (4) Moreover, gven that ths condton s satsfed, the couplng gan choce (6) reduces to c =. (43) Λ +Λ mn max Proof: For graphs havng a spannng tree wth pnnng nto the root node, and all egenvalues of Γ real, one has < Λ. ote that n that case ϕ = k and mn k cos α max = r so that necessary condton (37) s satsfed. Furthermore nequaltes (38) become r + r < c <. Λ Λ k From ths, a necessary and suffcent condton for the exstence of a nonempty ntersecton of these ntervals s k r + r <, Λ Λ mn max where the extremal values Λ mn = Λk, Λ mn max = Λ k Λ max mn,max = Λ k are the same for mn,max every k and are determned as n the proof of Lemma.9. 33

47 he sought nterval s gven as c r + r, Λ Λ mn max. hs nterval mples condton (4). herefore, the suffcent condton (4) s equvalent to (4). Examnng the postons of egenvalues on the real axs t s found that for the mnmal coverng crcle (, ) C r c one has So c Λ r c c + Λ mn max =, max mn = Λ = Λ =. max mn Λ Λ Λ r max mn = < Λ + Λ c max mn r. Λ ote that n the case of real egenvalues rato c r of all coverng crcles C ( c, r ) Λ k ther mnmal coverng crcle has also the mnmal. hs shows that (4) expresses the suffcent condton (4) specalzed to Γ havng all real egenvalues. heorem. then also gves the choce of c (43). hs specal case concernng real egenvalues of the graph matrx Γ bears strong resemblance to the case presented and studed n 4, concernng undrected graphs. For, note that f one uses the non-weghted protocol, u = ckε, nstead of (4), then, the egenvalues Λ k used n the analyss of ths chapter are those of ( L + G), not Γ n (). However, the condton that the egenvalues of ( L + G) be real s related to the graph beng undrected. It s noted that, even f all egenvalues of ( L + G) are real, the egenvalues of Γ may be complex, and vce versa. he mportance of weghtng s dscussed n Lemma.. 34

48 he radus r n (3) for Rccat-based desgn s mportant snce t s nstrumental n determnng suffcent condton (4) n heorem. as well as n Lemma.8. In the case of sngle-nput systems the expresson (3) smplfes. Remark.. If the node dynamcs () are sngle-nput, defne r by (3) wth the choce of * * Q P A P A * * A P BB P A = +, where * B P B * P > s a postve defnte soluton of the Rccat equaton * * A P BB P A * * * P A P A+ = B P B +. (44) hen u where ( A) r =. (45) u λ u ( A) λ are the unstable egenvalues of the system matrx A ndexed by u. hs follows from heorem. of 6. amely puttng Q as defned nto () yelds the soluton P * = P. In case of sngle-nput systems n B R and u s a scalar, so the result n 6 apples. ote that f the soluton to (44) s only postve sem-defnte, a regularzng term must be added to (44) as s done n 6. ote that * P defned as a soluton to Rccat equaton (44) becomes the soluton of () for the choce of Q gven n Remark.. hs means, n the context of ths chapter, that H - type Rccat feedback gan for that specfc choce of Q, appled to sngle-nput systems, maxmzes the value of r consdered here to be the radus of C (, r ) n the complex plane, rather than just the real nterval 6. he followng remark combnes the above results. Remark.3. If systems () are sngle-nput and the Γ matrx of the graph G ( V, E ) has all egenvalues real, selectng Q as n Remark., gves the condton (4) n the form 35

49 u max mn λ ( A). (46) u Λ + Λ < Λ Λ max Moreover ths condton s necessary and suffcent for synchronzaton for any choce of the feedback matrx K f all the egenvalues of A le on or outsde the unt crcle. Suffcency follows by Corollary. and Remark., assumng the condtons of heorem. hold. ecessty follows from 4. he next result shows another case where a smplfed form of r n heorem. can be gven. It follows drectly from smplfcaton of (3). Corollary.. Let matrx B be nvertble and Q max mn = I, then one has r =. (47) σ ( A PA) /.9 umercal Example hs example shows the mportance of weghtng by ( ) u = c + d + g Kε n protocol (4), and features unstable ndvdual agent dynamcs. he graph s drected and connected. he Laplacan L and the pnnng gans matrx G are gven as 3 3, G = dag(3,,,, 3). L = 3 he ndvdual agent dynamcs are gven by (), wth ( A, B) n controllable canoncal form, A =, B =.... he egenvalues of A are eg ( A) =.9,.434,. 434, therefore the uncontrolled systems are unstable. 36

50 Fgure. shows the non-weghted graph egenvalues, that s the egenvalues of ( L + G), and the weghted counterparts, that s the egenvalues of ( I D G) ( L G) Γ = + + +, wth the synchronzng regon C(, r ) also dsplayed. Clearly the non-weghted graph egenvalues do not satsfy the suffcent condton, (4) whle the weghted ones do. herefore the protocol u = ckε cannot be guaranteed to yeld synchronzaton, whle the weghted protocol (4) does. It s nterestng that the non-weghted graph egenvalues are real, whle the weghted ones may be complex. (a) (b) Fgure. (a) on-weghted graph egenvalues and ther coverng crcle. (b) Weghted graph matrx egenvalues, ther coverng crcle, and the synchronzaton regon (dashed crcle). he gan matrx K for the protocols (4) was desgned usng heorem.. Relevant values for the desgn are r =.583, r =.3333, c =.6684, r c = < r. We selected Q =.I3. Fgure.3 shows the dynamcs of a mult-agent system n the case that the couplng gan c s mproperly chosen. Synchronzaton s not acheved. 37

51 (a) (b) (c) Fgure.3 State trajectores of 5 agents; (a) Frst, (b) Second and (c) hrd state component, for an mproper choce of c. Smulated trajectores wth the couplng gan chosen by (5), (6) are depcted n Fgure.4. All nodes synchronze to the control node trajectory.. x. x (a) (b). 4 x (c) Fgure.4 State space trajectores of 5 agents; (a) Frst, (b) Second and (c) hrd state component, for proper value of c. 38

52 . Concluson hs chapter provdes condtons for achevng synchronzaton of dentcal dscrete tme state space agents on a communcaton graph structure. he concept of dscrete synchronzng regon n the z-plane s used. wo condtons for synchronzaton are gven whch decouple the graph propertes from the feedback desgn detals. he frst result, based on an H -type Rccat nequalty, gves a mlder condton for synchronzaton n terms of a radus that s generally dffcult to compute. he second result s n terms of a radus easly computed from an H -type Rccat equaton soluton, but may gve too conservatve a condton. Both are shown to yeld known results n the sngle-nput case. Example s gven to llustrate the proposed feedback desgn and smulatons justfy the used approach. 39

53 CHAPER 3 DISRIBUED OBSERVAIO AD DYAMIC OUPU COOPERAIVE FEEDBACK 3. Introducton he last two decades have wtnessed an ncreasng nterest n mult-agent network cooperatve systems, nspred by natural occurrences of flockng and formaton formng n nature. In the techncal world these systems are appled n formatons of spacecrafts, unmanned aeral vehcles, moble robots, dstrbuted sensor networks etc. 9. Early work wth networked cooperatve systems n contnuous and dscrete tme s presented n,3,4,7. hese papers referred to consensus wthout a leader,.e. the cooperatve regulator problem, where the fnal consensus value s determned solely by the ntal condtons. ecessary and suffcent condtons for the dstrbuted systems to synchronze are gven n 5,6,. On the other hand by addng a leader that pns to a group of agents one can have synchronzaton to a command trajectory through pnnng control, 7,3, for all ntal condtons. We term ths the cooperatve tracker problem. An elegant approach for nvestgatng suffcent condtons for a system to synchronze s provded by the concept of synchronzaton regon, 6,8,. hs concept allows for the control desgn problem to be decoupled from the graph topology, yeldng smplfed formulatons of suffcent condtons for synchronzaton to the leader trajectory. For contnuous-tme systems t s shown n 9,, that, by usng state feedback derved from the Rccat equaton, synchronzaton can be acheved for a broad class of communcaton graphs. Synchronzaton n contnuous tme usng dynamc compensators or output-feedback s consdered n,,4,4. For dscrete-tme cooperatve systems such general results are stll lackng, although there are references to specal cases of sngle-nput systems and undrected graph topology, 5, and ts extenson to drected graphs, 6. hese results were orgnally nspred by the earler work of 5,6, concernng optmal logarthmc quantzer densty for stablzng dscrete-tme systems. 4

54 In ths chapter we are concerned wth systems of ndvdual agents descrbed by dentcal lnear tme-nvarant dscrete-tme dynamcs. he graph s assumed drected, of fxed topology, and contanng a drected spannng tree. Contrary to the case presented n 6 and 8, perfect nformaton on the state of the neghbourng systems s not presumed. Output measurements are assumed and cooperatve observers are specally desgned for the multagent systems. Potental applcatons are dstrbuted observaton, sensor fuson, dynamc output regulators for synchronzaton, etc. For the needs of consensus and synchronzaton control we employ the cooperatve tracker, or pnnng control, 7,3,6. he key dfference between systems n contnuous-tme,,9, and dscrete-tme, 5,6,8 s n the form of ther stablty regon. More precsely, n contnuous-tme the stablty regon, as the open left-half s-plane, s unbounded by defnton, so the synchronzng regon can also be made unbounded. On the other hand, the dscrete-tme stablty regon, as the nteror of the unt crcle n the z-plane, s nherently bounded and, therefore, so are the synchronzng regons. hs makes condtons for achevng dscrete-tme stablty more strct than the contnuous-tme counterparts. Condtons for cooperatve observer convergence and for synchronzaton of the multagent system are shown to be related by a dualty concept for dstrbuted systems on drected graphs. It s also shown that cooperatve control desgn and cooperatve observer desgn can both be approached by decouplng the graph structure from the desgn procedure by usng Rccat-based desgn. Suffcent condtons are derved that guarantee observer convergence as well as synchronzaton. hs dervaton s facltated by the concept of convergence regon for a dstrbuted observer, whch s analogous, and n a sense dual, to the synchronzaton regon defned for a dstrbuted synchronzaton controller. Furthermore, the proposed observer and controller feedback desgns have a robustness property lke the one orgnally presented n 8 for controller desgn. hs chapter s organzed as follows. Secton 3. to 3.4. detal a local neghborhood observer structure. heorem 3. gves a Rccat-based desgn method that decouples the 4

55 desgn of the observer gans from the graph topology. he theorem gves suffcent condtons for the convergence of the proposed dstrbuted observer desgn. Secton 3.5 demonstrates that the dstrbuted observer and controller schemas are dual to one another by usng the concept of the reverse graph. Secton 3.6 presents three observer/controller regulator archtectures, together wth proofs guaranteeng that the agents synchronze and the observer errors converge to zero. hs demonstrates a separaton prncple for desgn of dynamc output feedback regulators on graph topologes. umercal examples are gven n Secton 3.7. and conclusons are presented n Secton Cooperatve Observer Desgn In ths secton we defne cooperatve observers on the nteracton graph G = (, ) V E and provde a desgn method for the observer gans that guarantees convergence of the observer errors to zero f certan condtons hold. he agents are coupled on a graph G = (, ) V E and have dynamcs gven as the dscrete-tme systems x y ( k +) = Ax ( k) + Bu ( k) ( k) = Cx ( k) (48) A control or leader node has the command generator dynamcs ( + ) = ( ) ( ) = ( ). x k Ax k, y k Cx k (49) n m p he state, nput, and output vectors are x, x R, u R, y, y R. ote that matrx A may not be stable. hs autonomous command generator descrbes many reference trajectores of nterest ncludng perodc trajectores, ramp-type trajectores, etc. 4

56 3.3 Observer Dynamcs It s desred to desgn cooperatve observers that estmate the states x ( ) measurements of the outputs ( ) j local output error k gven y k. Gven an estmated output, yˆ = Cxˆ, at node, defne the y = y yˆ. o construct an observer that takes nto account nformaton from the neghbors of node, defne the local neghborhood output dsagreement ( ) ( ) ε = e y y + g y y (5) o j j j where the observer pnnng gans are, g, wth g > only for a small percentage of nodes that have drect measurements of the output error ỹ of the control node. Wth the global state vector defned as n x = x x R, global output p y = y y R, the global o output dsagreement error ε ( k ) p R s ( k ) ( L G) I y ( k ) ( L G) I y ( k ) o ε = + m + + m. (5) hroughout ths chapter t s assumed that x = xˆ y,.e. the control node state s accurately known. hen, defne the dstrbuted observer dynamcs o ( ) ˆ ( ) ( ) ( ) ε ( ) xˆ k + = Ax k + Bu k c + d + g F k. (5) where F s an observer gan matrx and c a couplng gan. hs algorthm has local neghborhood output dsagreement (5) weghted by ( + d + ). he mportance of usng g ths weghtng s shown n Example 3.. he global dstrbuted observer dynamcs then has the form ( ) ( ) xˆ ( k + ) = I Axˆ ( k) + I Bu( k) + c I + D + G L + G Fy ( k), 43

57 ( ) ( ) ˆ ( ) ( ) xˆ ( k + ) = I A c I + D + G L + G FC x( k) + I Bu( k) + c I + D + G L + G Fy( k), where G dag ( g g ) =,, s the dagonal matrx of pnnng gans. Defnng the -th state observer error η ( k) = x ( k) xˆ ( k), n the global observer error η = η η R has the dynamcs η ( k + ) = I A c ( I + D+ G) ( L+ G) FC η ( k) Denote the global observer system matrx as. (53) A = I A c I + D + G L + G FC. (54) o ( ) ( ) Defnton 3.: he matrx appearng n (53) and (54) s referred to as the graph matrx ( I D G) ( L G) Γ = (55) he egenvalues Λ, k =..., of Γ are nstrumental n studyng the stablty of systems on k the graph G. Lemma 3.. If there exsts a spannng tree n the communcaton graph G wth pnnng nto a root node, then L + G, and hence Γ, s nonsngular, and both have all ther egenvalues n the open rght half plane. Proof:, Lemma Desgn of Observer Gans It s now desred to desgn the observer gans F to guarantee stablty of the observer errors and hence convergence of the estmates xˆ ( k ) to the true states x ( k ). Unfortunately, the stablty of the observer error dynamcs depends on the graph topology through the graph matrx egenvalues, Λ k, k =..., as shown by the followng result. 44

58 Lemma 3.. he global observer error dynamcs (53) are asymptotcally stable f and only f ( A c FC) ρ Λ < for all egenvalues Λ k, k k =... of the graph egenvalue matrx Γ = ( I + D + G) ( L + G). o Proof: Perform on (53) a state-space transformaton z = ( I ) η, where s a matrx n satsfyng Γ = Λ wth Λ beng a block trangular matrx. he transformed global observer error system s o ( ) [ ] ( ) o z k I A c FC z k + = Λ (56) herefore, (53) s stable f and only f I A cλ FC s stable. hs s equvalent to matrces A cλ kfc beng stable for every k snce Λ k are the dagonal elements of matrx Λ. he necessary condton for the stablty of (53) s detectablty of ( A, C ). Stablty of Λ mples detectablty of (, ) A c k FC A C. Lemma 3. s dual to a result n 5. Also, compare to. hs result s not convenent to use for the desgn of the observer gans F because t ntermngles the effects of the observer dynamcs and the graph matrx egenvalues. It s desred to provde here an observer gan desgn method that s ndependent of the graph egenvalues the development of such a desgn technque. Λ k. he followng notons facltate Defnton 3.: A coverng crcle (, ) C c r of the graph matrx egenvalues Λ k, k =..., s a closed crcle n the complex plane centered at c R and contanng all graph matrx egenvalues, Λ k, k =..., some possbly on the boundary. he noton of synchronzaton regon, 3, 6,,, has proven mportant for the desgn of state feedback gans for contnuous-tme systems that guarantee stablty on arbtrary graph 45

59 topologes. he synchronzaton regon was defned for dscrete-tme systems n 8. he next defnton provdes a dual concept for observer desgn. Defnton 3.3: For a matrx pencl A sfc { } such that S s ρ ( A sfc ) o = C <., s C, the convergence regon s a subset, So C, he convergence regon mght not be connected, but t s ntmately related to the complex gan margn regon defned next. Defnton 3.4: he complex gan margn U o C for some stablzng observer gan F, gven a { } system ( A, C) s a smply connected regon n C, U s ρ ( A sfc) s =. ote that for a stablzng observer gan F one has a relaton U o S. o o = C < contanng Lemma 3.3. Matrces A cλ k FC are stable for all egenvalues Λ f and only f k c Λk So; k. Proof: follows from Defnton 3.3 and 3.4. he man result of ths secton s a desgn technque for the dstrbuted observer gan F that does not depend on the graph egenvalues and guarantees observer stablty under a certan condton. It s presented n the followng theorem. heorem 3.. Rccat Desgn of Dstrbuted Observer Gans. Gven systems (48) assume the nteracton graph contans a spannng tree wth at least one pnnng gan nonzero that connects nto the root node. Let P > be a soluton of the dscrete-tme observer Rccat equaton, ( ) APA P + Q APC CPC CPA =, (57) where Q = Q >. Choose the observer gan matrx as Defne ( ) F APC CPC =. (58) 46

60 ( ( ) ) / / robs : σ Q APC CPC CPA Q = max /. (59) hen the observer error dynamcs (53) are stable f there exsts a coverng crcle (, ) C c r of the graph matrx egenvalues Λ, k =... such that k If (6) s satsfed then takng the couplng gan r < r obs. (6) c c = (6) c makes the observer error dynamcs (53) stable. he next techncal lemma s needed n the proof of heorem 3.. Lemma 3.4. he convergence regon, S o, for the observer gan F gven n heorem 3.., contans the crcle C (, r obs ). Proof: he observer Rccat equaton (57) can be wrtten as Select any symmetrc matrx P >. hen ( ) APA P + Q F CPC F =. (6) * ( ) ( ) A sc F P A sc F P = APA P s FCPA sapc F + s sfcpc F * * = APA P Re sapc ( CPC ) CPA + s APC ( CPC ) CPA = APA P (Re s s ) FCPC F = APA P ( s ) FCPC F (63) A suffcent condton for the stablty of ( A sfc) s that ths be less than zero. Insertng (6) ths s equvalent to Q + s FCPC F <. (64) 47

61 hs equaton furnshes the bound Furthermore, expressng F as (58) one obtans hs s an open crcle ( ), obs s <. (65) max σ max / / ( ( Q F CPC ) F Q ). (66) s < = r obs σ ( ( ) ) / Q APC CPC CPA Q / C r, and for s C n that crcle ρ( A sc F ) <, but ths also means that ρ( A sfc) < snce transposton does not change the egenvalues of a matrx. he guaranteed crcle of convergence C (, r obs ), beng smply connected n the complex plane and contanng s = s contaned wthn the observer gan margn regon U o. Proof of heorem 3.: Gven C (, r obs ) contaned n the convergence regon S o of matrx pencl A sfc, and the propertes of dlaton (homothety), assumng there exsts a drected spannng tree n the graph wth a nonzero pnnng gan nto the root node, t s clear that synchronzaton s guaranteed f all egenvalues Λ are contaned n a crcle (, ) k C c r smlar wth respect to homothety to a crcle concentrc wth and contaned wthn C(, r obs ). Homothety, as understood n ths chapter, refers to a radal projecton wth respect to the orgn,.e. zero n complex plane. hat s, t must be possble to brng C ( c, r ) nto ( ) same constant multpler. C r by scalng the rad of all graph egenvalues by the, obs he center of the coverng crcle c can be taken on the real axs due to symmetry and the radus equals r = max Λk c. akng these as gven, t s straghtforward that one should have k 48

62 r r obs c <. If ths equaton s satsfed then choosng c = / c maps wth homothety the coverng crcle of all egenvalues C ( c, r ) nto a crcle (, ) C r c concentrc wth and, for r > r / c, contaned n the nteror of the crcle C (, r obs ). ote that n (57) there s a condton P >, and one must have C of full row rank. A necessary condton for the exstence of a soluton P > to (57) s detectablty of (, ) A C. For the sake of computatonal smplcty, the coverng crcle C(, r obs ) s used to prove suffcency. If, however, ths coverng crcle s found not to satsfy (6), there could be other coverng crcles satsfyng ths condton, e.g. those havng smaller r c rato. he followng lemma hghlghts the robustness property of the proposed dstrbuted observer desgn. It formalzes the fact that under the suffcent condton of heorem 3. observer convergence s robust to small changes of the couplng gan value. Lemma 3.5. If the suffcent condton (6) of heorem 3. s satsfed then there exsts an open nterval of values for the observer couplng gan c guaranteeng convergence, obtaned as an ntersecton of a fnte famly of open ntervals, as n Chapter, 8. he value gven by (6) s contaned n all the ntervals, whence t s contaned n ther ntersecton. Proof: he same as for the analogous result of Chapter. 3.5 Dualty Between Cooperatve Observer and Controller Desgn In ths secton a dualty property s gven for cooperatve observers and cooperatve controllers on communcaton graphs, where c = c = c. It proves to be convenent to consder non-weghted observer and controller protocols gven by x ( k + ) = Ax ( k) + cbk ej ( xj x ) + g ( x x ) ( k), (67) j 49

63 xˆ ( ) ˆ k Ax ( k ) Bu ( k ) cf + = + ej ( y j y ) + g ( y y ) ( k ). (68) j he next result detals the dualty property for the dstrbuted controllers (67) and observers (68). For ths result, t s requred that the nteracton graph of the observer be the same as the nteracton graph for the control desgn, as assumed throughout ths chapter, but n addton n needs to be balanced. heorem 3.. Consder a networked system of dentcal lnear agents on a balanced communcaton graph G wth dynamcs ( A, B, C ) gven by (48). Suppose the feedback gan K n protocol (67) stablzes the global trackng error dynamcs (8) havng closed-loop system matrx A ( ) c = I A c L + G BK. (69) hen, assumng c = c = c, the observer gan F = K n protocol (68) stablzes the global observer error dynamcs (53) havng closed loop observer matrx ( ) Ao = I A c L + G FB, (7) for a networked dual system of dentcal lnear agents ( A, C, B ) on the reverse communcaton graph G '. Proof: Smlar to the proof gven n 3 for contnuous-tme systems. hough weghtng s not used n ths secton the mportance of weghtng s detaled n Example 3., subsecton hree Regulator Confguratons of Observer and Controller In ths secton are presented three dfferent methods of connectng observers and controllers at node nto a cooperatve dynamc output feedback regulator for cooperatve trackng. he observer graph and controller graph need not be the same. In the case that the observer and controller graph are not the same the suffcent condtons of heorem. of 5

64 Chapter and heorem 3. of Chapter 3 apply separately to the respectve graphs. he followng results are derved under a reasonable smplfyng assumpton of havng those two graphs equal, but apply smlarly to the more general case. he development of ths secton s a dscrete-tme verson of the contnuous-tme results of 3. he cooperatve regulators should have an observer at each node and a control law based on the observed outputs, and the separaton prncple should hold. hs s the case for the followng three system archtectures Dstrbuted Observers and Controllers In ths dynamc regulator archtecture, the controller uses a dstrbuted feedback law where ( ) u = c + d + g Kε, (7) ˆ j ( xˆ xˆ ) + g ( xˆ xˆ ) ˆ ε = e j j, (7) s the estmated local neghborhood trackng error. he observer s also dstrbuted and of the form (5). hus, both the controller and the observer depend on the neghbor nodes. he global state and estmate dynamcs are gven as ( ) ( ) ( ) ( ˆ ) x( k + ) = I A x( k) c I + D + G L + G BK x x ( k) (73) ( ) ( ) ( ) ˆ( ) ( ) ( ) xˆ k + = I A c I + D+ G L+ G FC x k + c I + D+ G L + G Fy( k) ( ) ( ) ( ) c I + D+ G L+ G BK xˆ x ( k) (74) ote that the assumpton, ˆx = x, was used n (73). he global state error then follows from ( + ) = ( + + ) ( + ) + ( + + ) ( + ) ( η + ) x k I A c I D G L G BK x( k) c I D G L G BK x ( k), whch further yelds the global error dynamcs and ( k + ) = ( I A c ( I + D + G) ( L + G) BK) ( k ) + c ( I + D + G) ( L + G) BK ( k ) A δ ( k ) B η ( k ), δ δ η = + c c (75) 5

65 η he entre error system s then = ( k + ) = ( I A c ( I + D + G) ( L + G) FC) η ( k ) A η ( k ) o δ η A B δ + = A o η c c ( k ) ( k ). (76). (77) Under the condtons detaled n heorem. of Chapter and heorem 3. of Chapter 3, the error system, (77), s asymptotcally stable, mplyng δ, η asymptotcally. hs guarantees synchronzaton of all the agents' states to the control node state Local Observers and Dstrbuted Controllers In ths archtecture, the controller s the dstrbuted form (7), (7). On the other hand, the observers are now local ( ) ˆ ( ) ( ) ( ) xˆ k + = Ax k + Bu k + Fy k. (78) and do not use nformaton from the neghbors. hen, the state observaton error η = x xˆ dynamcs s or globally η ( k ) = ( A FC) η ( k ) +, η ( k ) = I n ( A FC ) η ( k ) +. (79) hen, the overall error dynamcs are δ η ( k + ) Ac Bc δ = ( k) I ( A FC). (8) η ow, the observer gan F s easly selected usng, for nstance, Rccat desgn so that ( A FC) s asymptotcally stable. he feedback gan s selected by heorem 3.. hen, under the hypotheses of heorem 3., synchronzaton s guaranteed. 5

66 3.6.3 Dstrbuted Observers and Local Controllers In ths archtecture the controller s local of the form u = Kxˆ, (8) whch does not depend on the neghbors. On the other hand, the observer s dstrbuted and gven by protocol (5), wth ( ) ( ) ε = e y y + g y y. o j j j ote that nstead of ỹ n (5), whch s assumed dentcally equal to zero, here one uses the control node output y. In global form ths yelds x( k + ) = I ( ) ˆ Ax k I BKx( k). (8) xˆ ( k + ) = I ( A BK) xˆ ( k) + c Γ FCη ( k) c Γ FCx ( k) Expressng x, x ˆ through x, η = x xˆ gves x( k + ) = I ( A BK ) x( k) + I BKη( k), η( k + ) = I ( ) ( ) ( ) ( ) ˆ A BK x k + I BKη k I A BK x( k) cγ FCη ( k) + cγ FCx ( k) (83) = I ( A BK) η( k) + I BKη( k) c Γ FC( η x )( k) = ( I A c Γ FC) η( k) + c Γ FCx ( k) = A η( k) + c Γ FCx ( k), o Global trackng error dynamcs now follows as δ ( k + ) = ( I A) δ ( k) ( I BK ) xˆ ( k). (84) Usng that xˆ = x η = x x η + x = δ ( η x ) one fnds δ ( k + ) = I ( A BK ) δ ( k) + ( I BK )( η( k) x ( k)). (85) ether (83) nor (85) are autonomous systems snce an exogenous nput s present n form of control node state x ( k ). However, f one looks at the dynamcs of ϑ( k) = η( k) x ( k) t follows that ϑ( k + ) = A ϑ( k), and δ ( k + ) = I ( A BK) δ ( k) + ( I BK) ϑ( k). Or, more clearly n matrx form o 53

67 ( ) δ I A BK I BK δ ( k ) ( k) ϑ + = A o ϑ. (86) he control gan K s easly selected usng, e.g. Rccat desgn, so that ( A BK ) s asymptotcally stable. he observer gan s selected by heorem 3.. hen, under the hypotheses of heorem 3., synchronzaton, δ, s guaranteed. ote that the observer n (46) s based snce ϑ mples η x. hus, the observers effectvely estmate the trackng errors, convergng to xˆ ( k) = x ( k) x ( k) = δ ( k). Remark 3.: he three proposed observer/controller archtectures dffer n the amount of nformaton that must be exchanged between neghbors for observaton and control. When both the observer and controller are dstrbuted, each agent requres a sgnfcant amount of computaton and communcaton to produce the estmate of ts own state, snce neghbor outputs need to be measured and state,.e. output, estmates need to be communcated between neghbors. Also the control nput requres all the estmated neghbor states. Local observers and dstrbuted controller archtecture requre less computatons and communcaton, snce the state of each agent s estmated usng only ts own nputs and outputs, and only the state estmates need to be communcated between neghbors. he archtecture usng dstrbuted estmaton and local controller s nterestng because for control purposes agents do not need to communcate wth ther neghborhoods, though communcaton s needed for dstrbuted estmaton. he specfc egenvalues of A BK, A FC, A, A that appear n the augmented state equatons (77), (8), (86) determne the tme constants n each of the three archtectures. o c 54

68 3.7 umercal Examples 3.7. Example hs example shows the mportance of usng the weghtng n observer algorthm (4) and the control law (). Consder a set of 5 agents wth dynamcs gven by (48) wth ( A, B, C ) n controllable canoncal form A =, B =, C = [ ].... he control node has the dynamcs (49). he control node s pnned nto two of the nodes, and the graph structure s gven by 3 3 L = 3, G = dag(3,,,,3). (a) (b) Fgure 3. (a) on-weghted graph egenvalues and ther coverng crcle. (b) Weghted graph matrx egenvalues, ther coverng crcle, and convergence regon of observer (dashed crcle). 55

69 Fgure 3.a shows the non-weghted graph egenvalues, that s, the egenvalues of ( L + G), along wth ther coverng crcle. Fgure 3.b shows the magnfed part of Fgure 3.a contanng the weghted counterparts, that s the egenvalues of ( I D G) ( L G) Γ= + + +, ther coverng crcle and the observer convergence regon C (, r obs ) dsplayed n dashes. Dashed lnes markng a sector n complex plane depct the regon wheren the egenvalue coverng crcles should le so that they be scalable to the observer convergence regon. It can be verfed that the non-weghted egenvalues n Fgure 3.a do not satsfy the suffcent condton (6) gven n heorem 3., whle the weghted egenvalues n Fgure 3.b do. It s nterestng that the non-weghted egenvalues are real, whle the weghted egenvalues here are complex Example hs example gves a set of numercal smulatons for the three dfferent observercontroller archtectures n Secton 3.6. he dynamcs and the communcaton graph used are the same as n Example. Example a. Perfect state measurements In ths smulaton t s assume that perfect state measurements are avalable. hat s, the cooperatve control law s used assumng full state feedback, as n Chapter. hs provdes a baselne for comparson of the performance of the three dynamc regulator desgns gven n Secton 3.6. Fgure 3. shows the state components of the 5 nodes and the leader control node. Fgure 3.3 shows the frst components of the state trackng errors. All nodes synchronze to the state of the leader. 56

70 F rs t s ta t e c o m p o n e n t -p e rfe c t s ta te m e a s u re m e n ts S econd state com ponent -perfect state m easurem ents (a) (b) h r d s t a t e c o m p o n e n t - p e rfe c t s t a t e m e a s u r e m e n t s (c) Fgure 3. States of the nodes and the control node wth perfect state measurements (a) Frst state components, (b) Second state components, (c) hrd state components..8 ra c k n g erro r Fgure 3.3 Frst state trackng error component wth perfect state measurements. Example b. Dstrbuted observers and controllers hs smulaton s for the case of dstrbuted observers and controllers gven as desgn 3.6. n Secton 3.6. Fgure 3.4 depcts the frst state components of all 5 agents and the control node, as well as the frst components of the trackng errors δ and the frst components of the observer errors η. Synchronzaton s acheved. 57

71 .5 Frst state component-dstrbuted observer and controller.8 Frst trackng error component-dstrbuted observer and controller (a) (b). 6 F rs t o b s e rve r e r ro r c o m p o n e n t - d s t rb u t e d o b s e r ve r a n d c o n t ro lle r (c) Fgure 3.4 Dstrbuted observers and controllers. (a) Frst state components, showng synchronzaton, (b) Frst trackng error components, (c) Frst observer error components. Example c. Local observers and dstrbuted controllers hs smulaton s for the case of local observers and dstrbuted controllers gven as desgn 3.6. n Secton 3.6. Fgure.5 depcts the frst state components of all 5 agents and the control node, as well as the frst components of the trackng errors δ and the frst components of the observer errors η. Synchronzaton s acheved. 58

72 .5 Frst state component-local observer and dstrbuted controller.8 Frst trackng error component-local observer and dstrbuted controller (a) (b).6 Frst observer error com ponent-local observer and dstrbuted controller (c) Fgure 3.5 Local observers and dstrbuted controllers. (a) Frst state components, showng synchronzaton, (b) Frst trackng error components, (c) Frst observer error components. Example d. Dstrbuted observers and local controllers hs smulaton s for the case of dstrbuted observers and local controllers gven as desgn n Secton 3.6. Fgure 3.6 depcts the frst state components of all 5 agents and the control node, as well as the frst components of the trackng errors δ and the frst components of the observer errors η. Synchronzaton s acheved. ote that the observer errors n Fgure 3.6c do not converge to zero snce the observers estmate the trackng errors, convergng to xˆ ( k) = x ( k) x( k) = δ ( k). 59

73 .5 Frst state component-dstrbuted observer and local controller.8 Frst trackng error component-dstrbuted observer and local controller (a) (b). 5 F rs t obs erver error c om ponent -ds trbuted obs erver and loc al c ontroller (c) Fgure 3.6 Dstrbuted observers and local controllers. (a) Frst state components, showng synchronzaton, (b) Frst trackng error components, (c) Frst observer error components. Remark 3.: It s nterestng to compare the tme constants of state synchronzaton and estmate convergence for the three dynamc regulator archtectures. he observatons are n accordance wth the remark made after Secton 3.6. Compared to perfect state measurement, dstrbuted observers and controllers took almost twce as long for state synchronzaton, and almost the same for estmate convergence alone. he case of dstrbuted controller and local observer needed the shortest tme for state synchronzaton, shorter even than n case of local controller and dstrbuted observer. For the system consdered, t appears to be more advantageous to make observaton faster by usng a local observer, then to smplfy the control law. Dependng on the specfc egenvalues, perhaps for some other system t would pay more to use a local controller wth dstrbuted observer. 6

74 3.8 Concluson Algorthms that do not depend on the topologcal propertes of the graph were gven to desgn observer gans and feedback control gans for cooperatve systems on drected communcaton graphs. Rccat desgn for the observer and controller gans guarantees stablty f a certan condton n terms of the graph egenvalues holds. A dualty prncple for dstrbuted systems on balanced graphs was gven. hree cooperatve regulator desgns were presented, each usng cooperatve observers and feedback controllers, whch guarantee synchronzaton of mult-agent systems to the state of a control node usng only output feedback. Furthermore, the separaton prncple s shown to be vald n all three cases for mult-agent systems on graphs. hese results are an extenson of results for contnuous-tme systems n, 3, and, by dualty, of the results obtaned for dstrbuted feedback control systems n 8. Presented results could be extended to tme-varyng graphs n future work. 6

75 CHAPER 4 OPIMAL DISRIBUED COROL 4. Introducton he last two decades have wtnessed an ncreasng nterest n mult-agent network cooperatve systems, nspred by natural occurrences of flockng and formaton formng. Early work wth networked cooperatve systems n contnuous and dscrete tme s presented n,5,3,4,7,6. Early work generally referred to consensus wthout a leader, where the fnal consensus value s determned solely by the ntal condtons. We term ths the cooperatve regulator problem. ecessary and suffcent condtons for the dstrbuted systems to synchronze were gven. On the other hand, by addng a leader that pns to a group of other agents one can have synchronzaton to a command trajectory through pnnng control, 5, 3, 8 for all ntal condtons. We term ths the cooperatve tracker problem. It s shown n, 3 that, by usng state feedback derved from the local algebrac Rccat equaton, synchronzaton can be acheved for a broad class of communcaton graphs. Synchronzaton usng dynamc compensators or output feedback s consdered n 9,3,5. Cooperatve optmal control was recently consdered by many authors- 33,34,36,37,38,39, to name just a few. Optmalty of a control protocol gves rse to desrable characterstcs such as gan and phase margns, that guarantee robustness n presence of some types of dsturbances, 4,4. he common dffculty, however, s that n the general case optmal control s not dstrbuted, 34,36. Soluton of a global optmzaton problem generally requres centralzed,.e. global, nformaton. In order to have local dstrbuted control that s optmal n some sense t s possble e.g. to consder each agent optmzng ts own, local, performance ndex. hs s done for recedng horzon control n 33, mplctly n 3, and for dstrbuted games on graphs n 35, where the noton of optmalty s ash equlbrum. In 37 the LQR problem s phrased as a maxmzaton problem of LMI's under the constrant of the communcaton graph topology. hs s a constraned optmzaton takng nto account the local 6

76 character of nteractons among agents. It s also possble to use a local observer to obtan the global nformaton needed for the soluton of the global optmal problem, as s done n 34. In the case of agents wth dentcal lnear tme-nvarant dynamcs, 38 presents a suboptmal desgn that s dstrbuted on the graph topology. Optmal control for mult-agent systems s complcated by the fact that the graph topology nterplays wth system dynamcs. he problems caused by the communcaton topology n the desgn of global optmal controllers wth dstrbuted nformaton can be approached usng the noton of nverse optmalty, 4. here, one chooses an optmalty crteron related to the communcaton graph topology to obtan dstrbuted optmal control, as done for the sngle-ntegrator cooperatve regulator n 36. hs connecton between the graph topology and the structure of the performance crteron allows for the dstrbuted optmal control. In the case that the agent ntegrator dynamcs contans topologcal nformaton, 39 shows that there s a performance crteron such that the orgnal dstrbuted control s optmal wth respect to t. In ths chapter are consdered fxed topology drected graphs and lnear tme-nvarant agent dynamcs. Frst, theorems are provded for partal stablty and nverse optmalty of a form useful for applcatons to cooperatve control, where the synchronzaton manfold may be noncompact. In our frst contrbuton, usng these results, we solve the globally optmal cooperatve regulator and cooperatve tracker problems for both sngle-ntegrator agent dynamcs and also agents wth dentcal lnear tme-nvarant dynamcs. It s found that globally optmal lnear quadratc regulator (LQR) performance cannot be acheved usng dstrbuted lnear control protocols on arbtrary dgraphs. A necessary and suffcent condton on the graph topology s gven for the exstence of dstrbuted lnear protocols that solve a global optmal LQR control problem. In our second contrbuton, we defne a new class of dgraphs, namely, those whose Laplacan matrx s smple,.e. has a dagonal Jordan form. On these graphs, and only on these graphs, does the globally optmal LQR problem have a dstrbuted lnear protocol 63

77 soluton. If ths condton s satsfed, then dstrbuted lnear protocols exst that solve the global optmal LQR problem only f the performance ndces are of a certan form that captures the topology of the graph. hat s, the achevable optmal performance depends on the graph topology. he structure of ths chapter s the followng; Secton 4. deals wth stablty of possbly noncompact nvarant manfolds, motvated by partal stablty. Secton 4.3 ntroduces condtons of optmalty and nverse optmalty and the connecton wth stablty. hese results are appled n Sectons 4.4 and 4.5. Secton 4.4 dscuses optmal leaderless consensus and pnnng control for sngle-ntegrator agent dynamcs, and Secton 4.5 gves the optmalty results for general lnear tme-nvarant agent dynamcs. In Sectons 4.4 and 4.5 t s found that optmalty for a standard LQR performance ndex s only possble for graphs whose Laplacan matrx satsfes a certan condton. In Secton 4.6 ths condton s further dscussed and shown to be satsfed by certan classes of dgraphs, specfcally for those graphs whose Laplacan matrx has a dagonal Jordan form. hs condton holds for undrected graphs. Conclusons are gven n Secton Asymptotc Stablty of the Consensus Manfold hs secton presents concepts of partal stablty appled to noncompact manfolds. he defntons are based on neghbourhoods of a manfold and avod the need for global coordnates, n contrast to usual formulaton of partal stablty, e.g. 4. he fact that a manfold to whch convergence s to be guaranteed s noncompact means that usual approaches for compact manfolds based on proper Lyapunov functons are not applcable as such. Furthermore when dealng wth noncompact manfolds one makes the dstncton between nonunform and unform stablty. In the cooperatve control consensus problem, where there are agents wth states x R, =..., t s desred for all agents to acheve the same state, that s x ( t) x ( t) j 64

78 as t,, j. hen the consensus manfold, S : span() noncompact. Let the system dynamcs be gven as x = f ( x, u) = f ( x) + g( x) u, 65 =, where [ ] = R, s where f ( x ), g( x ), and u( t ) are assumed to be such that the exstence of a unque soluton to the ntal value problem s guaranteed. Defnton 4.: Let S be a manfold embedded n a topologcal space X. A neghborhood D ( S) of S s an open set n X contanng the manfold S n ts nteror. Defnton 4.: Let S be a manfold embedded n a metrc space ( X, d ), and let D ( S) be a neghborhood of S. An ε -neghborhood of S D ( S) s defned as { ε } U ( S ) = x X d ( x, S ) <, where d( x, S): = nf d( x, y) s the dstance from x X to S as ε gven by the metrc d. y S ote that n the case of compact manfolds any neghborhood D ( S) contans some ε - neghborhood, but n the case of noncompact manfolds ths need not be true. For the needs of defnng stablty of noncompact manfolds, one uses neghborhoods that contan some ε - neghborhood. We call such neghborhoods regular. Defnton 4.3: A manfold S s sad to be (Lyapunov) stable f there exsts a regular neghborhood D ( S) of S, such that for every ε -neghborhood U ( S) contaned n t, there exsts a subneghborhood V ( S) satsfyng the property x() V( S) x( t) U ( S) t. If D ( S) can be taken as the entre space X, then the stablty s global. If S s Lyapunov stable and furthermore there exsts a neghborhood W ( S) of S satsfyng the property x() W ( S) d( x( t), S) as t, then S s asymptotcally stable. If d( x( t), S) we say that the trajectory x( t ) converges to S. ε ε

79 If S s Lyapunov stable and for every ε -neghborhood U ( S) of S, there exsts a subneghborhood V( S) U ( S) contanng a δ -neghborhood V ( S) then S s unformly ε stable. In ths case the stablty concluson can be phrased as ε >, δ > such that ( ( ) ) δ ( ( ) ) d x, S < d x t, S < ε t. If a manfold s unformly stable and asymptotcally stable so that some neghborhood W ( S) contans a δ -neghborhood W ( S) satsfyng the property x() W ( S) d ( x( t), S) as t, unformly on W ( S), then S s unformly δ asymptotcally stable. If a manfold s unformly asymptotcally stable and there exst constants K, σ > such that d( x( t), S) Kd( x(), S) e σt, for all x () n some δ -neghborhood W ( S) then the partal δ δ δ ε δ stablty s exponental. A suffcent condton for the varous types of stablty of a manfold, possbly noncompact, s gven as the followng theorem, motvated by 4, Chapter 4. heorem 4.. (Partal Stablty). Gven a manfold S, possbly noncompact, contaned n a neghborhood D S, f there exsts a C functon V : D R and a class K functon α such that V ( x) = x S V ( x) α ( d( x, S)) for V ( x) for x D x D \ S then S s Lyapunov stable. If furthermore there s a class K functon β such that α ( d( x, S)) V ( x) β ( d( x, S)) In ths context, unform convergence means ε >, > s. t. d( x( t), S) < ε t, where depends on ε and d ( x, S ), but not on x tself. 66

80 then the stablty s unform. If n addton there s a class K functon γ such that V ( x) < γ ( d( x, S)) then the stablty s unformly asymptotc. If there exst a, b, c, p > such that then the stablty s exponental. ε p ad( x, S) V ( x) bd ( x, S) V ( x) < cd ( x, S) p Proof: ake any U ( S), then on the boundary U ( S) one has d( x, S) = ε yeldng a bound V ( x) α ( ε ). akng η α ( ε ) that α( d( x, S)) = and defnng = { x ( S ) V ( x) < η} η ε p D D one has for x D η < η meanng that d ( x, S) < ε whch mples that D U ( S). akng x() D, owng to the nonncreasng property of V( x ), one obtans α ( d ( s( t), S) V ( x( t)) V ( x()) < α ( ε ), t, guaranteeng x( t) U ( S) t, provng the Lypapunov stablty of S. ε However there s no guarantee that, due to noncompactness, one does not have η ε η nf d ( x, S ) = x Dη whch means that theη level set of V( x ) s not bounded away from the manfold S. hs s remeded by the unformty requrement snce then one can choose δ > such that β ( δ ) = α ( ε ), and by the prevous argument for x() V ( S) α( d( s( t), S) V ( x( t)) V ( x()) < β ( δ ) = α( ε ) Unform asymptotc and exponental stablty follow from the proof analogous to partal stablty δ results n 4. he purpose of class K functons s to bnd the level sets of the Lyapunov functon away from the manfold and away from nfnty, allowng the nterpretaton of a value of the Lyapunov 67

81 functon as a measure of the dstance from the manfold. For global conclusons one would requre K, as an analog of properness,.e. radal unboudedness. Remark 4.: ote that the unformty property s always found n the case where the manfold S s compact. However, snce we are dealng here wth a noncompact manfolds, the dfference needs to be emphaszed. heorem 4. wll be used together wth the nverse optmalty approach of the next secton, to guarantee asymptotc stablzaton to a consensus manfold that s also optmal. Remark 4.: In case partal stablty condtons are satsfed one does not necessarly have bounded trajectores. rajectores are constraned to a noncompact set, and fnte escape to nfnty, whle remanng n the noncompact neghborhood of S, s thereby not precluded. herefore, n order to avod such an occurrence one needs to ndependently mpose that solutons can be extended for all tme. Snce lnear systems are globally Lpschtz solutons cannot escape to nfnty n fnte tme. However, n more general cases, wth nonlnear dynamcs, an addtonal, proper, Lyapunov functon could be used to guarantee boundedness of trajectores to a compact set. ote that ths does not contradct the stablty of a noncompact set, snce La Salle nvarance prncple can be used to guarantee convergence to (a subset of) such a set. ote that the requrements on the Lyapunov functon specalze n case of compact manfolds to famlar condtons, 4. In that case all bounded open neghborhoods are precompact. Specfcally when one consders an equlbrum pont the condtons specalze to { } V ( x) = x =, V ( x) α( x ) for x D \, V for x D. In case of lnear systems and pertanng quadratc partal stablty Lyapunov functons V( x) = x Px the target manfold S s the null space of the postve semdefnte matrx P. 68

82 Also the unformty condton s automatcally satsfed snce σ ( P) y y Py σ ( P) y mn > max, where y ker P, and y serves as the dstance from the null space. 4.3 Inverse Optmal Control hs secton presents nverse optmalty notons for specfc applcaton to the consensus problem. Inverse optmalty s a property of a stablzng control u = φ( x), e.g. guaranteed by a control Lyapunov functon V ( x ), that s optmal wth respect to some postve (sem) defnte performance ndex. Let the system be gven by the affne-n-control form x = f ( x, u) = f ( x) + g( x) u. (9) Certan results of optmal control can be explctly derved for systems n ths form, e.g. Hamlton-Jacob-Bellman equaton Optmalty he followng lemma detals the condtons for optmalty of a stablzng control under an nfnte horzon crteron, 4. Lemma 4.. (Optmalty) Consder the control affne system (9). Let S be a target manfold, possbly noncompact. Gven the nfnte horzon optmalty crteron J ( x, u) ( x, u) dt = L, (9) f there exst functons V (x), φ (x), and class K functons α, γ satsfyng the followng condtons V ( x) = x S V ( x) α( d( x, S)) φ( x) = x S V ( x) f ( x, φ( x)) γ ( d( x, S)) H ( x, φ ( x)) = H ( x, u) 69

83 wth H ( x, u) = L ( x, u) + V ( x) f ( x, u) then the feedback control u = φ( x) s optmal wth respect to the performance ndex (9) and asymptotcally stablzng wth respect to the target set S. Furthermore the optmal value of the performance ndex s J ( x, φ ( x)) = V ( x()). Condtons, and 4 show that V ( x ) s a control Lyapunov functon for the closed loop system guaranteeng asymptotc stablty of the set S, by heorem 4.. If S s taken as a pont,.e. a compact manfold of dmenson zero, then those condtons become the famlar condtons on the Lyapunov functon stablzng an equlbrum pont 4.3. Inverse Optmalty V () = V ( x) α ( x ) x φ () = V ( x) f ( x, φ( x)) <. If the feedback control u = φ( x) s asymptotcally stablzng, then there exsts a Lyapunov functon V( x ) satsfyng the condtons of Lemma 4. by the nverse Lyapunov theorems 4,4. In nverse optmalty settngs an asymptotcally stablzng control law u = φ( x) s gven. hen the performance ntegrand L ( x, u) and Lyapunov functon V (x) are to be determned. In ths work we are concerned wth nonnegatve performance ndex ntegrands, so ths s to be contrasted wth the more general nverse optmalty where the performance ntegrand need not be nonnegatve, as long as t guarantees a stablzng control. hat the performance ntegrand must be postve (sem-) defnte mposes constrants on V( x ). 7

84 Lemma 4.a. (Inverse optmalty) Consder the control affne system (). Let u = φ(x) be a stablzng control, wth respect to a manfold S. If there exst scalar functons V (x) and L ( ) satsfyng the followng condtons V ( x) = x S V ( x) α ( d( x, S)) L ( x) γ ( d( x, S)) L ( x) + V ( x) f ( x) V ( x) g( x) R g( x) V ( x) = (9) 4 ϕ ( x) = R g( x) V ( x) then u = φ(x) s optmal wth respect to the performance ndex wth the ntegrand x L ( x, u) = L ( x) + u Ru. Moreover the optmal value of the performance crteron equals J( x, φ ( x)) = V( x ). Proof: Assume one has an optmal control problem (9), (9) wth the optmalty performance ntegrand n (9) L ( x, u) = L ( x) + u Ru. hen Lemma 4. states the soluton of optmal problem can be obtaned by formng the Hamltonan H ( x, u) = L ( x) + u Ru + V ( x) ( f ( x) + g( x) u). hs gves the optmal control n form of u H ( x, u) = u R + V ( x) g( x) =, u = φ x = R g x V x ( ) ( ) ( ) hs feedback control vanshes on the set S snce the gradent V ( x) equals zero there. he Hamltonan evaluated at ths optmal control equals zero snce H ( x, φ( x)) = L ( x) + V ( x) f ( x) V ( x) g( x) R g( x) V ( x) =. 4 7

85 hs Hamltonan can then be concsely wrtten n quadratc form H ( x, u ) = L + u Ru + V f + V gu = L + V gr g V 4 = u R u V gr g V V gu = φ R = ( u φ ) R ( u φ ). φ = φ Ru he value of the performance crteron then follows as J = L( x, u) dt = V ( x) f ( x, u) dt + H ( x, u) dt. dv = dt + H ( x, u ) dt V ( x ) V ( x ( t )) H ( x, u ) dt dt = + he optmum s reached precsely at u = φ( x), for whch the ntegral on the rght sde vanshes, and by asymptotc stablty assumpton V( x( t)), thus the optmal value of the performance crteron equals ( ) = (, φ( )) = ( ). * J x J x x V x hat ths optmal feedback control u = φ( x) s stablzng follows also from the Lyapunov equaton ( ) = L + V gr g V V gr g V 4 = L ( x) V gr g V L ( x) γ ( d( x, S)) 4 V x = V f + V gu = V f V gr g V By the assumptons of Lemma 4.a the Lyapunov functon V ( x ) satsfes all the condtons of the heorem 4. for partal stablty. In the lnear quadratc case,.e. x = Ax + Bu, L = x Qx, V( x) = x Px, the Hamlton Jacob Bellman equaton (HJB) (3) becomes the Algebrac Rccat equaton (ARE) Q + A P + PA PBR 7 B P =. (93)

86 he next more general result allows state-control cross-weghtng terms n the performance ntegrand L. hs result s used n the followng chapter. Lemma 4.b (Inverse optmalty wth cross weghtng terms) Consder the control affne system (). Let u = φ(x) be a stablzng control, wth respect to a manfold S. If there exst scalar functons V ( x ) and L ( x), L ( x ) satsfyng the followng condtons V ( x) = x S V ( x) α( d( x, S)) L ( x) = x S L x V x f x L x V x g x R L x g x V x 4 ( ) + ( ) ( ) ( ) + ( ) ( ) ( ) + ( ) ( ) = φ ( x) = R ( L ( x) + g( x) V ( x)) L ( x) + L ( x) φ + φ Rφ γ ( d( x, S)) then u = φ(x) s optmal wth respect to the performance ndex wth the ntegrand L ( x, u) = L ( x) + L ( x) u + u Ru. Moreover, the optmal value of the performance crteron equals J( x, φ ( x)) = V( x ). Proof: Assume one has an optmal control problem (9), (9) wth the optmalty performance ntegrand n (9) L ( x, u) = L ( x) + L ( x) u + u Ru. hen Lemma 4. states the soluton of optmal problem can be obtaned by formng the Hamltonan H ( x, u) = L ( x) + L ( x) u + u Ru + V ( x) ( f ( x) + g( x) u). hs gves the optmal control n form of u H ( x, u) = u R + L ( x) + V ( x) g( x) = u = φ x = R L x + g x V x ( ) ( ( ) ( ) ( )) 73

87 hs feedback control vanshes on the set S snce the gradent V ( x) and L ( x ) equal zero there. he Hamltonan evaluated at ths optmal control equals zero snce H x x = L x + L x R L x + g x V x (, φ ( )) ( ) ( ) ( ( ) ( ) ( )) + R ( L ( x) + g ( x) V ( x)) R R ( L ( x) + g ( x) V ( x)) + V x f x + V x g x R L x + g x V x ( ) ( ) ( ) ( ) ( ( ) ( ) ( )) = L + V f L R L L R g V + L R L V gr g V L R g V V gr L V gr g V = L V f L R L L R g V V gr g V = L + V f L + V g R L + g V = 4 hs Hamltonan can then be concsely wrtten n quadratc form 4 H ( x, u) = L + L u + u Ru + = L + V gr g V + LR L + LR g V 4 4 = u Ru + V gr g V + LR L + LR g V L u + V gu = ( u φ ) R( u φ ). = φ Ru V f + V gu = φ R φ he optmal value of the performance crteron then follow as n Lemma 4.a. hat ths optmal feedback control u = φ( x) s stablzng follows also from the Lyapunov equaton 74

88 V x V f V gu V f V gr L g V = L + LR L + LR g V + V gr g V 4 4 V gr g V V gr L ( ) = + = ( + ) = L ( x) V gr g V + L R L 4 4 = L ( x) L ( x) φ φ Rφ γ ( d ( x, S )) By the assumptons of Lemma 4.b the Lyapunov functon V ( x ) satsfes all the condtons of the heorem 4. for partal stablty. 4.4 Optmal Cooperatve Control for Quadratc Performance Index and Sngle-ntegrator Agent Dynamcs hs secton consders nverse optmalty of consensus protocols for the leaderless and pnnng control cases, 3,4,8. We call these respectvely the cooperatve regulator and cooperatve tracker problem. hs secton consders only sngle ntegrator dynamcs x = u R, (94) or n global form x = u, (95) where x = [ x ] and [ ] x u u u =. More general dynamcs are consdered n the next secton Optmal Cooperatve Regulator In the leaderless consensus or cooperatve regulator problem, 3, where there are agents wth states x R, =.., t s desred for all agents to acheve the same state, that s x ( t) x ( t) as t, (, j). hen the consensus manfold, S : = span (), where j 75

89 [ ] = R, s noncompact. herefore to apply nverse optmalty to the cooperatve regulator problem, 3, one needs the partal stablty results of heorem 4.. Defne the local neghborhood error as η = ( ) e x x j j. (96) he overall local neghborhood error vector [ ] j = s equal to e = Lx. hen a local e η η votng protocol that guarantees consensus s gven as u = η,, or n global form u = Lx, (97) whch gves the global closed loop system x = Lx. (98) Lemma 4.3. If the graph has a spannng tree then s a smple egenvalue of the graph Laplacan matrx. Furthermore the control law u = Lx solves the cooperatve regulator problem,, 5, for the system (95). If the condton of Lemma 4.3 s satsfed then the measure of the dstance from the consensus manfold s gven by any norm of the local neghborhood dsagreement vector e = Lx. he followng theorem states suffcent condtons for optmalty of a dstrbuted control law. hs shows when the dstrbuted control (97) s globally optmal on the graph. heorem 4.. Let the system be gven as (95), then for some R = R > the control u = Lx s optmal wth respect to the performance ndex J ( x, u) = ( x L RLx + u Ru) dt = ( e Re + u Ru) dt, (99) 76

90 and s stablzng to a manfold whch s the null space of L f there exsts a postve semdefnte matrx P = P satsfyng P = RL. () Proof: Frst note that snce R s nonsngular the null space of P equals that of L. he Lyapunov functon V ( x) = x Px equals zero on the null space of P and s greater than zero elsewhere snce P. If one ntroduces the orthogonal complement of the null space of P one has V ( x) = x Px ( P) y where x = x + y, x ker P, y ker P. Snce y s a σ > mn measure of the dstance d( x, ker P ) the Lyapunov functon s found to satsfy the condtons of Lemma 4.. he part of the performance ntegrand L ( x) = x Qx = x L RLx ( L RL) y σ > mn satsfes the condton of heorem 4.. he Algebrac Rccat equaton (93) gven as s satsfed by P = RL = L R, and L RL PR P = u Lx R Px = =. herefore all the condtons of Lemma 4. are satsfed by ths lnear quadratc problem, whch concludes the proof. hs result was frst gven n 36. ote that the tme dervatve of Lyapunov functon V ( x) = x Px equals ( ) ( ) mn ( ), V x = x PL + L P x = x L RLx σ > L RL y guaranteeng asymptotc stablty of the null space of L or equvalently that of P. Also, snce σ > max V ( x) = x Px ( P) y, accordng to heorem 4., the stablty s unform. If the graph has a spannng tree, by Lemma 4.3, zero s a smple egenvalue of L and thus the null space of P s the consensus manfold span (). hen, consensus s reached. 77

91 Remark 4.3: he constrant RL = P s a jont constrant on the graph topology and performance crteron. hs can easly be met n the case of undrected graphs L = L by selectng R I =, n whch case P = L and Q = L L. he value of the optmalty crteron s then gven as * J ( x ) V ( x ) x Lx = = whch s the graph Laplacan potental, 3. Moreover σ ( L) n the > mn proof equals λ ( L), the Fedler egenvalue of the graph. Also the constrant () s seen to be met by detal balanced graphs wth R = Λ where Λ L = P s a symmetrc Laplacan matrx. For general dgraphs, snce P s symmetrc, the constrant () mples the condton More dscusson about condton () s gven n Secton Optmal Cooperatve racker RL = L R. () In case of the optmal cooperatve tracker for system (95) t s desred for all agents to synchronze to the state x of a leader node. hat s x ( t) x ( t),. It s assumed that the leader node has dynamcs x =. A dstrbuted control law that accomplshes ths s gven by u = e ( x x ) + g ( x x ) j j j () wth g the pnnng gans whch are nonzero only for a small number of agents. hs corresponds n global form to the dstrbuted tracker control law u = ( L+ G) δ, (3) where δ = x x s the global trackng dsagreement vector and G = dag( g g ). hs gves the global closed-loop dynamcs and the global closed-loop trackng dsagreement dynamcs system x = ( L+ G) δ, (4) δ = u = ( L + G) δ. (5) 78

92 hs must be asymptotcally stablzed to the orgn. hat means that partal stablty notons need not be used, and the sought soluton P of the Algebrac Rccat Equaton (93) must be postve defnte. Lemma 4.4. If the graph has a spannng tree, gven that there exsts at least one non zero pnnng gan connectng nto a root node, then L + G s nonsngular, and the control u = ( L + G) δ solves the cooperatve trackng problem, 9, for the system (95). If the condtons of Lemma 4.4 are satsfed, then the measure of dstance from the target state x s gven by any norm of the local neghborhood error vector e = ( L + G) δ. he next result shows when the dstrbuted control (3) s globally optmal on the graph. heorem 4.3. Let the error dynamcs be gven as (5), and the condtons of Lemma 4.4 be satsfed. hen for some R = R > the control u = ( L + G) δ s optmal wth respect to the performance ndex J ( δ, u) = ( δ ( L + G) R( L + G) δ + u Ru) dt = ( e Re + u Ru) dt, (6) and s stablzng to the reference state x f there exsts a postve defnte matrx P = P > satsfyng P = R( L + G). (7) Proof: he Lyapunov functon V( δ ) = δ Pδ >, and ( ) ( ) L ( ) δ = δ Qδ = δ L + G R L + G δ > satsfy the condtons of Lemma 4.. he Algebrac Rccat equaton ( L + G) R( L + G) PR P = s satsfed by P, and u ( L G) δ R P = + = δ thus provng the theorem. 79

93 Smlar remarks about the constrant between R and L + G apply here as n Remark 4.3. Under condtons of Lemma 4.4 and (7) P s nonsngular snce both R and L + G are. For undrected graphs, one choce that satsfes (7) s R = I. hen P = L + G. A more general choce s any R that commutes wth L + G. More dscusson about condton (7) s relegated to Secton 4.6. Remark 4.4: Both n the optmal cooperatve regulator and tracker performance crtera, the expressons Lx and ( L + G) δ, respectvely, play a promnent role. In ether case one can concsely wrte the performance ndex as (, ) ( J e u = e Re + u Ru) dt. 4.5 Optmal Cooperatve Control for Quadratc Performance Index and Lnearme-nvarant Agent Dynamcs hs secton consders nverse optmalty of consensus protocols for the leaderless and pnnng control cases for agents havng states n x R wth lnear tme-nvarant dynamcs n x = Ax + Bu R, or n global form x = ( I A) x + ( I B) u (8) 4.5. Optmal Cooperatve Regulator In the cooperatve regulator problem t s desred for all agents to acheve the same state, that s x ( t) x ( t) as t, (, j). Defne the local neghborhood error as j 8 η = ( ) e x x j j. (9) j n where η R. he overall local neghborhood error vector e = η η s equal to e = ( L I ) x. n

94 hen, a dstrbuted control that guarantees consensus s gven as u = ck η, for an approprate couplng gan c > and local feedback matrx K, as shown n 3. In global form ths s the local votng protocol u = c( L K ) x, () whch gves the global closed loop system x = ( I A cl BK ) x () he next result shows when the dstrbuted control () s globally optmal on the graph. heorem 4.4. Let the system be gven as (8). Suppose there exst a postve semdefnte matrx P = P, and a postve defnte matrx P = P > satsfyng P = cr L, () + + =, (3) A P P A Q P BR B P for some Q = Q >, R = R >, R = R > and a couplng gan c >. Defne the feedback gan matrx K as K = R B P. (4) hen the control u = cl Kx s optmal wth respect to the performance ndex J( x, u) = x c ( L K) ( R R )( L K) cr L ( A P + P A) x + u ( R R ) udt, (5) and s stablzng to the null space of L I for suffcently hgh couplng gan c satsfyng (7). n Proof: Form the matrx P = P P. Snce P s nonsngular the null space of P equals the null space of P I whch by nonsngularty of R s the same as that of n 8 L I. herefore by n

95 postve semdefnteness the Lyapunov functon ( ) V x = x Px = x ( P P ) x s zero on the null space of L I and postve elsewhere. By the arguments smlar to those presented n the n proof of heorem 4. ths Lyapunov functon satsfes the condtons of heorem 4.. Also, L ( x) = x Qx = x ( c ( L K ) ( R R )( L K ) cr L ( A P + P A)) x (6) equals zero on the null space of L I, and satsfes the condton of Lemma 4. f the value of n the couplng gan s taken as ( R L ( Q K R K )) c >. (7) σσ > ( ) max mn L R L K RK Snce Q = c L K R R L K cr L A P + P A ( ) ( )( ) ( ) = + ( ), c L R L K RK cr L Q K RK one has c σ mn L R L K R K cσ max R L Q K R K Q > ( ) ( ( )) >, whch gves the lower bound on the couplng gan c. he algebrac Rccat equaton for system, (8), ( I A) P P( I A) Q P( I B) R ( I B) P + + =, wrtten as, P A P + P P A+ Q ( P P B) R R ( P B P ) = P ( A P + P A) + Q PR P ( P BR B P ) = s satsfed by the choce of P = P P snce Q = c L K R R L K crl A P + P A ( ) ( )( ) ( ) = c L R L K R K + cr L ( Q P BR B P ) = P R P P BR B P + P ( Q P BR B P ) = Q P BR B P + P ( Q P BR B P ), 8

96 where Q = c L R L = PR P s ntroduced for notatonal smplcty. It follows by the condtons (), (3) of the heorem that P ( A P + P A + Q P BR B P ) + ( Q PR P) ( P BR B P ) =. By condtons (), (3) and (4) the control satsfes u cl Kx R ( I B) Px = = =. ( R R )( I B )( P P ) x = R P R B P x herefore, all the condtons of the heorem 4. are satsfed, completng the proof. ote that the tme dervatve of the Lyapunov functon V ( x) = x Px equals V ( x) = x Px = x P( I Ax cl BK x) = x ( P ( P A+ A P ) P( I B)( R R )( I B ) P) x = x Q + c L K R R L K x x Qx ( ( ) ( )( )) mplyng asymptotc stablty to the nullspace of Q, whch equals the null space of L I (c.f. n (6)). If the condtons of Lemma 4.3 hold then ker L = span( ), and under the condtons of heorem 4.4 consensus s reached. Remark 4.5: he graph topology constrant () s smlar to the constrant (), and comments smlar to those stated n Remark 4.3 apply here as well. here exsts a P = P > satsfyng (3) f ( A, B ) s stablzable and ( A, Q ) s observable. he value of the couplng gan c must be suffcently great to overpower the, generally, ndefnte terms stemmng from the drft dynamcs as n condton (7). If the communcaton graph satsfes condtons of Lemma 4.3 the null space of L I equals the synchronzaton manfold S : = span( α), n therefore synchronzaton s asymptotcally acheved n the optmal way. n α R, 83

97 4.5. Optmal Cooperatve racker In the pnnng control or cooperatve tracker problem, 8,9 t s desred for all the agents to reach the state x of a leader or control node, that s x ( t) x ( t) as t,. It s assumed that x = Ax. Consder the optmal cooperatve tracker problem for the system (8) and defne the local neghborhood trackng error as ( ) ( ) e x x g x x j j (8) j η = + n where η R. he pnnng gans g are nonzero only for a few nodes drectly connected to a leader node. he overall local neghborhood trackng error s equal to e = ( L + G) Inδ, where the global dsagreement error s δ = x x. hen a dstrbuted control that guarantees synchronzaton s gven as u = ck η, for an approprate couplng gan c > and local feedback matrx K. In global form ths s u = c( L+ G) K δ, (9) wth G = dag ( g... g ) the dagonal matrx of pnnng gans. hs gves the global dynamcs o acheve synchronzaton the global dsagreement system or, wth control u = c( L+ G) Kδ, x = I Ax c( L + G) BK δ. () δ = ( I A) δ + ( I B) u, () δ = ( I A c( L + G) BK ) δ, () must be asymptotcally stablzed to the orgn. hs means that partal stablty notons need not be used, and the sought soluton P of the Algebrac Rccat Equaton (93) must be postve 84

98 defnte. Stablty of () to the orgn s equvalent to asymptotc reference trackng and synchronzaton. hs dsagreement system can be stablzed usng the dstrbuted control law (9), 3. he next result shows when the dstrbuted control (9) s globally optmal on the graph. Conclusons of Lemma 4.a allow for the followng result stated as a theorem. heorem 4.5. Let the error dynamcs be gven as (), and condtons of Lemma 4.4 be satsfed. Suppose there exst a postve defnte matrx P = P >, and a postve defnte matrx P = P > satsfyng P = cr ( L + G), (3) A P P A Q P BR B P + + =, (4) for some Q = Q >, R = R >, R = R > and a couplng gan c >. Defne the feedback gan matrx K as K = R B P. (5) hen the control u = c( L + G) Kδ s optmal wth respect to the performance ndex = J( δ, u) δ c (( L G) K ) ( R R )(( L G) K ) cr ( L G) ( A P P A) δ u ( R R ) udt, (6) and s stablzng to the orgn of () for suffcently hgh couplng gan c satsfyng (7). Proof: Form the matrx P = P P. Snce P and P are nonsngular so s P. he Lyapunov functon V( δ) = δ Pδ >, by the arguments smlar to those presented n the proof of heorem 4.3. hs Lyapunov functon satsfes the condtons of heorem 4., specalzed to a sngle equlbrum pont. Also, L ( δ ) = δ Qδ = δ [ c (( L G ) K ) ( R R )(( L G ) K ) cr ( L G ) ( A P P A)] δ 85

99 s postve defnte and satsfes the condton of heorem 4. f the value of the couplng gan s taken as Snce one has σ σ ( R ( L + G) ( Q K R K )) c >. (7) σ (( ) ( ) ) max mn L + G R L + G K RK = (( + ) ) ( )(( + ) ) ( + ) ( + ) Q c L G K R R L G K cr L G A P P A = ( + ) ( + ) + ( + ) ( ) c L G R L G K RK cr L G Q K RK (( + ) ( + ) ) σ ( ( + ) ( )) > >, c mn L G R L G K RK c max R L G Q K RK Q whch gves the lower bound on the couplng gan c. he Algebrac Rccat equaton (93) for system (), ( I A) P P( I A) Q P( I B) R ( I B) P + + =, wrtten as P A P + P P A+ Q ( P P B) R R ( P B P ) = P ( A P + P A) + Q PR P ( P BR B P ) =, s satsfed by the choce of P = P P snce Q = c L + G K R R L + G K cr L + G A P + P A (( ) ) ( )(( ) ) ( ) ( ) = c ( L + G) R ( L + G) K R K + cr ( L + G) ( Q P BR B P ) = P R P P BR B P + P ( Q P BR B P ) = Q P BR B P + P ( Q P BR B P ) where Q = c ( L + G) R ( L + G) = PR P s ntroduced for notatonal smplcty. It follows by the condtons (3), (4) of the theorem that P ( A P + P A + Q P BR B P ) + ( Q P R P ) ( P BR B P ) = By condtons (3), (4) and (5), the control u satsfes 86

100 u = c L + G K = R I B Pδ ( ) δ ( ) = ( R R )( I B )( P P ) δ = R P R B P δ. herefore, all the condtons of the heorem 4. are satsfed, completng the proof. ote that the tme dervatve of the Lyapunov functon V ( δ ) = δ Pδ equals V ( δ ) = δ P δ = δ P( I A c( L + G) BK ) δ = δ ( P ( P A + A P ) P( I B)( R R )( I B ) P) δ = δ Q + c L + G K R R L + G K δ δ Qδ < ( (( ) ) ( )(( ) )), mplyng asymptotc stablty of () to the orgn. Remark 4.6: he graph topology constrant (3) s smlar to the constrant (7), and comments smlar to those stated there apply here as well. hs constrant s further dscussed n Secton 4.6. here always exsts a P P = > satsfyng (4) f ( A, B ) s stablzable and ( A, Q ) observable. he value of the couplng gan c must be suffcently great to overpower the, generally, ndefnte terms stemmng from the drft dynamcs 4.6 Constrants on Graph opology Ax, as n condton (7). In ths secton we ntroduce a new class of dgraphs whch, to our knowledge, has not yet appeared n the cooperatve control lterature. hs class of dgraphs admts a dstrbuted soluton to an approprately defned global optmal control problem. he essental condtons for global optmalty of the dstrbuted control (97) or (3) for the cooperatve regulator and the cooperatve tracker, respectvely, are respectvely () and (7). Both of these have the same form nvolvng ether the Laplacan matrx, L, or the pnned graph Laplacan, L + G. hs secton nvestgates classes of graphs that satsfy those condtons. Generally one can express these condtons as RL = P, (8) 87

101 where R = R >, L s a sngular (.e. the graph Laplacan) or nonsngular (.e. the pnned graph Laplacan) M-matrx, and P = P, a sngular or nonsngular postve defnte matrx. Equvalently RL = L R (9) For the followng classes of graph topologes one can satsfy ths condton Undrected Graphs Gven that the graph s undrected, then L s symmetrc,.e. L = L so the condton (8) becomes a commutatvty requrement See Remark 4.3, where t s shown that the choce R undrected graphs. hen P = L. RL = L R = LR. (3) = I always satsfes ths condton for More generally, condton (3) s satsfed by symmetrc matrces R and L f and only f R and L have all egenvectors n common. Snce L s symmetrc t has a bass of orthogonal egenvectors, and one can construct R satsfyng (3) as follows. Let be an orthogonal matrx whose columns are egenvectors of L, then L = Λ wth Λ a dagonal matrx of real egenvalues. hen for any postve defnte dagonal matrx Θ one has that R = Θ > commutes wth L and satsfes the commutatvty requrement (3). ote that the R so constructed depends on all the egenvectors of the Laplacan L n (8) (.e. the graph Laplacan or the pnned Laplacan as approprate) Detaled Balanced Graphs Gven a detal balanced graph (cf. Secton.), ts Laplacan matrx s symmetrzble, that s there exsts a postve dagonal matrx Λ > such that L = Λ P where P a symmetrc graph Laplacan matrx. hen, condton (8) holds wth R = Λ. Recall that for detaled 88

102 balanced graphs, the dagonal elements of R n the leaderless case are then the elements of the left egenvector of the Laplacan for the egenvalue of zero Drected Graphs wth Smple Laplacan In ths secton we ntroduce a new class of dgraphs whch, to our knowledge, has not yet appeared n the cooperatve control lterature. hs class of dgraphs admts a dstrbuted soluton to an approprately defned global optmal control problem. Gven a drected graph, let t be such that Laplacan matrx L (ether the graph Laplacan or the pnned graph Laplacan) n (8) s smple,.e. there exsts a bass of egenvectors so that ts Jordan form s dagonal. hen the Laplacan matrx s dagonalzable, so there exsts an nvertble matrx such that whence t follows that L L = Λ = Λ = L, L = L. = Λ s dagonal. hen one has that herefore, R = = R > satsfes the condton (9), 45. ote that ths R depends on all the egenvectors of the Laplacan L n (8). hs dscusson motvates the followng theorem. heorem 4.6. Let L be a Laplacan matrx (generally not symmetrc). hen there exsts a postve defnte symmetrc matrx R = R > such that RL = P s a symmetrc postve semdefnte matrx f and only f L s smple,.e. there exsts a bass of egenvectors of L. Proof: () Let L be smple. hen t s dagonalzable,.e. there exsts a transformaton matrx such that L = Λ, where Λ s a dagonal matrx of egenvalues of L. hen L = Λ = Λ = L, mplyng L L, whch mples that ( ) L L ( ) = =. Let R =. Obvously, R = R > and P = RL = L = Λ snce Λ ( x x Px = x Λ x = y Λy y),

103 condton RL () Let L be a Laplacan matrx. Suppose there exsts R = R > satsfyng the = P s a symmetrc postve semdefnte matrx. hen one needs to show that L s smple. o show ths, we wll prove the contrapostve by contradcton. So we suppose the negaton of the contrapostve. hat s, suppose L s not smple but that there exsts R = R > satsfyng the condton RL = P s a symmetrc postve semdefnte matrx. Snce L s not smple, there exsts a coordnate transformaton brngng L to a Jordan canoncal form =, L J wth nonzero superdagonal (otherwse L would be smple). hen one has = = = =. But then ( R ) J = J ( R ). herefore there exsts RL RJ P P J R R = R = R > such that R J = J R. Wthout loss of generalty let us assume that the frst Jordan block s not smple, and wth a slght abuse of notaton R, wll refer to the correspondng block n R.hen one has that R ( λi + E) = ( λi + E ) R R E = E R,,,,, where E s a nlpotent matrx havng ones on the superdagonal. hs dentty means that the frst row and frst column of R, are zero, except for the last entry. However then R cannot be postve defnte, snce there are vanshng prncpal mnors (Sylvester's test). Snce the smplcty of the Laplacan matrx s a necessary and suffcent condton for the topology constrant (8) to be satsfed, one can state the followng theorem relatng the graph Laplacan matrx (unpnned or pnned) to the optmalty of decentralzed cooperatve control as gven n prevous sectons of ths chapter. heorem 4.7. he dstrbuted control n cooperatve regulator and tracker problems for sngle ntegrator agent dynamcs (95) s optmal under optmalty crteron 9

104 for some R > smple. (, ) ( J e u = e Re + u Ru) dt,, wth pertanng error functons, f and only f the Laplacan matrx n (8) s Proof: If the control law s optmal then the constrant (8) on P, R, L s satsfed, hence, by heorem 4.6, L s smple. If, conversely L s smple, then there s a quadratc performance crteron wth respect to whch the cooperatve dstrbuted control gven by L s nversely optmal. Remark 4.7: hese results place condtons on the graph topology that guarantee that the cooperatve stablzng dstrbuted control s optmal wth respect to a structured performance crteron, whch n turn guarantees asymptotc consensus or synchronzaton. he followng secton wll allow for a more general form of performance crteron whch yelds optmal control for general drected graphs. 4.7 Concluson hs chapter presents nverse optmalty of consensus and pnnng control algorthms n cases of sngle-ntegrator and general lnear tme-nvarant agent dynamcs. A partal stablty motvated result on stablty of noncompact nvarant manfolds s used n optmalty and nverse optmalty condtons to guarantee optmalty of cooperatve control achevng asymptotc consensus or synchronzaton under a postve semdefnte performance ntegrand. he optmalty requrement mposes constrant between communcaton graph topology and the structure of performance ntegrand. 9

105 CHAPER 5 OPIMAL COOPERAIVE COROL FOR GEERAL DIGRAPHS: PERFORMACE IDEX WIH CROSS-WEIGHIG ERMS 5. Introducton When dealng wth general drected graphs the constrant () on the graph Laplacan L, or (7) on the pnned graph Laplacan L + G, s too restrctve. hese constrants only hold for a specal class of dgraphs whose Laplacan matrx (or pnned Laplacan L + G ) has dagonal Jordan form. hs constrant can be relaxed, and optmal cooperatve controllers developed for arbtrary dgraphs, by allowng state-control cross-weghtng terms n the performance crteron, 44. hen, requrng that the performance crteron be postve (sem) defnte leads to condtons on kernel matrx P n V ( x) = x Px, or equvalently on the control Lyapunov functon, whch should be satsfed for the exstence of dstrbuted globally optmal control law. hs condton s mlder than the condtons (), (7) where no cross-weghtng term s allowed n the performance ndex. In the followng subsectons we treat the optmal cooperatve regulator and tracker for sngle-ntegrator dynamcs, ncludng a state-control cross weghtng term n the performance ndex. In Secton 5.4. we dscuss the resultng constrant condtons on the graph topology that must be satsfed by the graph Laplacan for exstence of an optmal controller of dstrbuted form. It s shown that, unlke condtons () and (7), these new condtons can be satsfed for arbtrary drected graphs, contanng a spannng tree, by proper selecton of the performance ndex weghtng matrces. Sectons 5.5 and 5.6 treat the optmal cooperatve regulator and tracker for lnear tmenvarant agent dynamcs. Agan, t s shown that, f cross-weghtng terms are allowed n the performance ndex, then the condtons requred for exstence of a dstrbuted optmal controller can be satsfed on arbtrary dgraphs. Concluson s provded n secton

106 5. Optmal Cooperatve Regulator- Sngle-ntegrator Agent Dynamcs hs secton consders the cooperatve regulator problem for sngle-ntegrator agent dynamcs x = u or n global form (95). Snce the consensus manfold s noncompact, t s necessary to use the partal stablty results n heorem 4. Usng the dstrbuted control protocol u = Lx gves the closed-loop system x = Lx. he conclusons of Lemma 4.b allow for the followng result. heorem 5.. Let the system be gven as (95). hen for some R = R > the control = L wth u = φ( x) = Lx s optmal wth respect to the performance ndex J ( x, u) ( x, u) dt the performance ntegrand L ( x, u) = x L RLx + u Ru + x ( L R P) u, and s stablzng to a manfold whch s the null space of of L f there exsts a postve semdefnte matrx P = P, havng the same kernel as L, satsfyng the nequalty condton L P PL PR P +. (3) Proof: Let V ( x) = x Px be a partal stablty Lyapunov functon. hen V x x PLx x PL L P x x PR Px σ PR P y ( ) = = ( + ) ( ) > mn where x = x + y, y ker P performance ntegrand equals. hus the control s stablzng to the null space of P. Also the L L RL L R P ( x, u) = x u RL P R u. x 93

107 One has L ( x, u) f R > and the Schur complement nequalty (3) holds. he control s optmal snce u = φ x = R L + g V ( ) ( ) = + = R (( RL P) x Px) R ( RL) x = Lx. he performance ntegrand evaluated at the optmal control satsfes L( x, φ( x)) = x L RLx + x LRLx x ( L R P) Lx = x L RLx x L RLx + x PLx = + x ( PL L P) x x PR Px σ ( PR P) mn y Hence, all the condtons of Lemma 4.b of Ch. 4 are satsfed, whch concludes the proof. If the graph has a spannng tree, then by Lemma 4.3, zero s a smple egenvalue of L and thus the null space of L s the consensus manfold span (). hen, consensus s reached by all agents. ote that f the constrant P = RL n () s met as n prevous sectons, the crossweghtng term n L ( x, u) n the proof vanshes, so that and the performance crteron s quadratc and therefore L ( x, u). hat s, substtutng P = RL nto (3) shows that (3) s satsfed. Remark 5.: he kernels of P and L n (3) need not be the same, but under condton (3) ther relaton s gven by the followng result. Proposton 5.. Let there exst a symmetrc postve semdefnte P satsfyng (3) for some L, and R > then ker L ker P. 94

108 he proof of ths proposton uses a result on symmetrc postve semdefnte matrces summarzed n the followng lemma, whch s not dffcult to prove. Lemma 5.. Gven a postve semdefnte symmetrc matrx Q the kernel of Q equals the set where the pertanng quadratc form satsfes x Qx =. ote that ths result, albet straghtforward, s not trval snce a nonsymmetrc matrx Q generally does not share ths property. ow, to prove the proposton, notce that under (3) + s a L P PL PR P symmetrc postve semdefnte matrx. Consder the quadratc form ( + ). x L P PL PR P x Assume x ker L but x ker P, then for such an x x L Px x PLx x PR Px x PR Px Px R Px + = ( ) ( ). = However, snce R > ths forces Px =, whence t follows x ker herefore, one has x ker L x ker P meanng ker L ker P, whch s a contradcton. P, whch proves the proposton. More dscusson about the condton (3) s provded n Secton Optmal Cooperatve racker- Sngle-ntegrator Agent Dynamcs In case of the optmal cooperatve tracker for the sngle-ntegrator agent dynamcs system (95), one has the global dsagreement error dynamcs (5), whch must be stablzed to the orgn. hs means that partal stablty notons need not be used, and the sought matrx P must be postve defnte. herefore, for the optmal cooperatve tracker problem the appled logc s the same, the only dfference beng that postve semdefnteness s replaced by postve defnteness n the performance ndex and Lyapunov functon. 95

109 heorem 5.. Let the error dynamcs be gven as (5), and the condtons of Lemma 4.4 be satsfed. hen for some R = R > the control u = φ( δ) = ( L + G) δ s optmal wth respect to the performance ndex J ( δ, u) ( δ, u) dt = L wth the performance ntegrand ( L ( δ, u) = δ L + G) R( L + G) δ + u Ru + δ (( L + G) R P) u >, and s stablzng the orgn for (5) f there exsts a postve defnte matrx P = P, satsfyng the followng nequalty ( L G) P P( L G) PR P > (3) Proof: Let V ( δ ) = δ Pδ be a Lyapunov functon. hen V P L G P L G L G P PR P ( ) ( ) ( ( ) ( ) ) δ = δ + δ = δ δ < δ δ <. hus the control s stablzng to the orgn. Also the performance ntegrand equals L( δ, u) = L + G R L + G L + G R P δ R( L + G) P R u [ δ u ] ( ) ( ) ( ) >, and L ( x, u ) > f R > and nequalty (3) holds (by Schur complement). he control s optmal snce u = φ δ = R L + g V ( ) ( ) = R (( R( L + G) P) δ + Pδ ) = R ( R( L + G)) δ = ( L + G) δ. he performance ntegrand evaluated at the optmal control satsfes 96

110 ( ( L( δ, φ( δ )) = δ L + G) R( L + G) δ + δ L + G) R( L + G) δ δ (( L + G) R P)( L + G) δ = xδ ( L + G) R( L + G) δ δ ( L + G) R( L + G) δ + δ P( L + G) δ = δ δ > δ δ > ( P( L G) ( L G) P) PR P. Hence all the condtons of Lemma 4.b of Ch. 4 are satsfed, whch concludes the proof. ote that f the constrant P = R( L + G) n (7) s met as n prevous sectons, the cross-weghtng term n L ( δ, u) n the proof vanshes, so that the performance crteron s quadratc and therefore L ( x, u ) >. hat s, P = R( L + G) guarantees that (3) s satsfed. More dscusson about the condton (3) s provded n Secton Condton for Exstence of Dstrbuted Optmal Control Wth Cross-Weghtng erms n the Performance Index Condtons (3) and (3), whch allow state-control cross-weghtng terms n the performance ndex, express jont constrants on the graph matrces L, or L + G, and the choce of matrx R that are less strct than the condtons () and (7), respectvely, that guarantee the exstence of an optmal controller of dstrbuted form when no cross-weghtng term s allowed. In partcular, note that f condton () or (7) hold then, respectvely, (3) and (3) hold as well, and the cross-weghtng terms equal zero. he addtonal freedom stemmng from allowng the presence of cross-weghtng terms n the performance ntegrand allows for more general graph topologes excluded under the Rccat condtons () or (7) to be nverse optmal. Condtons () and (7) are only satsfed by dgraphs that have a smple Jordan form for L or L + G, respectvely. hs ncludes undrected graphs and the detal balanced dgraphs. It s now shown that condtons (3) and (3) can be satsfed for arbtrary dgraphs. For arbtrary dgraphs, a matrx P havng the same kernel as L and satsfyng the optmal cooperatve regulator condton (3) can be constructed as follows. o fnd 97

111 P = P such that + and ker P = ker L one can solve the equvalent L P PL PR P nequalty by solvng an ntermedate Lyapunov equaton L P PL PR P, L P PL = Q PR P, wth Q = Q, ker Q ker L s stablzng to ts nullspace, and equals =. he soluton P to ths Lyapunov equaton exsts snce L. L τ Lτ P = e Qe dτ hat P = P follows from the constructon, and f L has a smple zero egenvalue, whch s necessary for consensus, ker P = ker Q = ker L. Snce P s bounded for any fxed Q, and the soluton P does not depend on R, choosng R bg enough, n the sense of any matrx norm, wll make PR P Q. And such P satsfes condton (3). A smlar method can be used to construct, for any dgraph, the requred matrx P for the optmal tracker condton (3). 5.5 General Lnear me-nvarant Systems- Cooperatve Regulator hs secton deals wth agents havng states n x R wth drft dynamcs x = Ax + Bu L I n n R, or n global form (8). Snce the synchronzaton manfold, null space of, s noncompact, the partal stablty heorem 4. must be used. heorem 5.3. Let the system be gven as (8). Defne the local feedback matrx to be K such that the cooperatve feedback control u = cl K x, wth a scalar couplng gan c >, 98

112 makes (8) asymptotcally converge to the consensus manfold,.e. acheve consensus. hen there exsts a postve semdefnte matrx P = P, satsfyng c L K R L K P I A I A P ( ) ( ) ( ) ( ) (33) P cl BK I A + cl BK I A P P I B R I B P (34) ( ( )) ( ( )) ( ) ( ) for some R = R >. And the control u = cl K x s optmal wth respect to the performance ndex J ( x, u) ( x, u) dt = L wth the performance ntegrand L( x, u) = x c ( L K ) R( L K ) P( I A) ( I A ) P x + u Ru + x c( L K ) R P( I B) u, and s stablzng to the null space of P for suffcently hgh couplng gan c. Proof: Snce the cooperatve feedback s assumed to make (8) synchronze the closed loop system matrx A = I A cl BK cl defnes a partally stable system wth respect to the consensus manfold. hs manfold equals the kernel of L I f L contans a spannng tree. n hen for any postve semdefnte matrx Q = Q, havng the kernel equal to the consensus manfold, there exsts a soluton P = P to the Lyapunov equaton cl cl PA + A P = Q A cl τ A cl τ = τ P e Qe d wth kernel equal to the consensus manfold. 99

113 he nequalty (33) s satsfed for suffcently hgh values of the couplng gan c >, and nequalty (34) s satsfed va Lyapunov equaton by choosng R = R > suffcently bg, n the sense of matrx norms, such that ( ) ( ) P I B R I B P Q. Let V ( x) = x Px be a partal stablty Lyapunov functon. hen V ( x) = x Px = x P(( I A) x c( L BK ) x) ( ( ) ( ) ) ( ) ( ) [ ( ) ( ) ] [ + ] = cl cl x P( I B) R ( I B ) Px = x P I A + I A P x cx P L BK x cx L BK Px = x P I A cl BK + I A cl BK P x = x PA A P x x Qx he quadratc form x Qx serves as a measure of dstance from the consensus manfold. hus the control s stablzng to the consensus manfold, as assumed. Also the performance ntegrand equals L( x, u) = [ x u ] c ( L K ) R( L K ) P( I A) ( I A ) P c( L K ) R P( I B) x cr( L K ) ( I B) P R u and L ( x, u) f nequaltes (33) and (34) hold (by Schur complement). he control s optmal snce u = φ x = R L + g V ( ) ( ) = R (( cr( L K) ( I B ) P) x + ( I B ) Px) = ( ( )) = ( ) R cr L K x c L K x he performance ntegrand evaluated at the optmal control satsfes

114 L( x, φ( x)) = x c ( L K ) R( L K ) P( I A) ( I A ) P x + x c ( L K ) R( L K ) x cx c( L K ) R P( I B) ( L K ) x [ ] [ ( ) ( ) ] ( ) [( ) ( ) ] [ ] cl = x P( I A) + ( I A ) P x + cx P( I B)( L K ) x = + + x P I A I A P x cx P L BK x = + x cl BK I A P P c = x A P PA x = x Qx x P( I B) R ( I B ) Px cl L BK I A x Hence all the condtons of Lemma 4.b of Chapter 4 are satsfed, whch concludes the proof. Specfyng the form of P and R as P = P P, R = R R and assumng further that K = R B P, where P s a soluton of local Rccat equaton P A A P Q P BR B P + + = as used n heorems 4 and 5,,3 the nequalty (34) becomes cpl P BK P P A + A P + cl P K B P PR P P BR B P. ( ) Equvalently, snce P BK = K B P = P BR B P = K R K, one has c PL + L P PR P K R K P K R K Q. ( ) ( ) ote the smlarty to condton (3) ntroduced n sngle ntegrator consensus problem Secton 5.. If condton (3) s satsfed then for suffcently hgh value of the couplng gan c ths constrant can be met. Clearly f Rccat condtons (), (3) are satsfed then the cross-weghtng term vanshes cr L K = I B P c L K = R I B P. ( ) ( ) ( ) ( )

115 One should also note that choosng K = R B P affords an nfnte nterval of postve values for couplng gan c that acheve synchronzaton, 3, whch allows one to fnd a suffcently hgh value of c to satsfy nequalty (33) wthout worryng about loosng synchronzaton. he same apples whenever one needs to choose c to be, for any purpose, suffcently bg. If the communcaton graph s connected, then by Lemma 4.3 the null space of L I n equals the synchronzaton manfold S : = span( α), asymptotcally acheved n the optmal way. n α R, therefore synchronzaton s 5.6 General Lnear me-nvarant Systems- Cooperatve racker If the goal s to synchronze dynamcs (8) to the leader s trajectory x = Ax,.e. solve the cooperatve trackng problem, then one should use the global error system δ = x x, wth the same global error dynamcs δ = ( I A) δ + ( I B) u. (35) heorem 5.4. Let the system be gven as (35). Defne the local feedback matrx to be K such that the cooperatve feedback control u = c( L + G) K δ, wth a scalar couplng gan c >, makes (35) asymptotcally converge to the orgn. hen there exsts a postve defnte matrx P = P >, satsfyng (( + ) ) (( + ) ) ( ) ( ) > c L G K R L G K P I A I A P (36) P c L + G BK I A + c L + G BK I A P P I B R I B P > (37) ( ( ) ( )) ( ( ) ( )) ( ) ( ) for some R = R >. And the control u = c( L + G) K δ s optmal wth respect to the performance ndex J ( δ, u) ( δ, u) dt = L wth the performance ntegrand

116 L u c L G K R L G K P I A I A P δ u Ru ( δ, ) = δ (( + ) ) (( + ) ) ( ) ( ) + + δ c(( L + G) K ) R P( I B) u >. Proof: Snce the cooperatve feedback s assumed to stablze (35) to the orgn the closed loop system matrx A = I A c( L + G) BK cl defnes a stable system. hen for any postve defnte matrx Q = Q >, there exsts a soluton P = P > to the Lyapunov equaton cl cl PA + A P = Q Acl τ Acl τ = τ > P e Qe d he nequalty (36) s satsfed for suffcently hgh values of the couplng gan c >, and nequalty (37) s satsfed va Lyapunov equaton by choosng R = R > suffcently bg, n the sense of matrx norms, such that ( ) ( ) P I B R I B P < Q. Let V ( δ ) = δ Pδ > be a Lyapunov functon. hen V ( δ) = δ P δ = δ P(( I A) δ c(( L + G) BK ) δ) = δ ( P( I A) + ( I A ) P) δ cδ P(( L + G) BK ) δ cδ (( L + G) BK ) Pδ ( PA A P) Q P( I B) R ( I B ) P. cl cl = δ ( P( I A c( L + G) BK ) + ( I A c( L + G) BK ) P) δ = δ + δ = δ δ < δ δ hus the control s stablzng to the orgn, as assumed. Also the performance ntegrand equals L( δ, u) = [ δ u] c (( L+ G) K ) R(( L+ G) K ) P( I A) ( I A ) P c(( L+ G) K ) R P( I B) δ cr(( L+ G) K ) ( I B ) P R u > and L ( δ, u) > f nequaltes (36), (37) hold (by Schur complement). he control s optmal snce 3

117 u = φ δ = R L + g V ( ) ( ) = R (( cr(( L + G) K) ( I B ) P) δ + ( I B ) Pδ ) = R ( cr(( L + G) K)) δ = c( L + G) K δ he performance ntegrand evaluated at the optmal control satsfes L ( δ, φ( δ)) = δ c (( L+ G) K ) R(( L+ G) K ) P( I A) ( I A ) P δ + δ c (( L+ G) K ) R(( L+ G) K ) δ cδ c(( L+ G) K ) R P( I B) (( L+ G) K ) δ [ P( I A) ( I A ) P] δ cδ P( I B)(( L G) K ) [ P( I A) ( I A ) P] c P(( L G) BK ) [( c( L G) BK I A) P P( c( L G) BK I A) ] [ A P PA ] Q P I B R I B P = δ = δ + δ + δ + δ = δ = δ δ = δ δ > δ ( ) ( ) δ cl cl Hence all the condtons of Lemma 4.b of Chapter 4 are satsfed, whch concludes the proof. δ δ Specfyng the form of P and R as P = P P, R = R R, and assumng further that K =, where P s a soluton of the local Rccat equaton R B P P A + A P + Q P BR B P =, as used n heorems 4.4 and 4-5 of Chapter 4, the nequalty (37) becomes cp L + G P BK P P A + A P + c L + G P K B P PR P P BR B P >. ( ) ( ) ( ) Equvalently, snce P BK = K B P = P BR B P = K R K, one has c P L + G + L + G P P R P K R K P K R K Q >. ( ( ) ( ) ) ( ) 4

118 ote the smlarty to condton (3) ntroduced n sngle-ntegrator consensus problem Secton 5.. If condton (3) s satsfed then for suffcently hgh value of the couplng gan c ths constrant can be met. Remark 5.: Due to the connecton of ths nequalty and the one n the earler secton wth nequaltes for sngle-ntegrator systems (3) and (3) respectvely, conclusons n Remark 5.. on constructng the sutable P apply here as well. Clearly f Rccat condtons (3), (4) are satsfed then the cross-weghtng term vanshes (( ) ) ( ) (( ) ) ( ) cr L + G K = I B P c L + G K = R I B P. One should also note that choosng K = R B P affords an nfnte nterval of postve values for couplng gan c that acheve stablzaton, 3, whch allows one to fnd a suffcently hgh value of c to satsfy nequalty (36) wthout worryng about stablty of the closed-loop system. 5.7 Concluson In order to lft the constrants on the graph topology detaled n Chapter 4 one can use a more general form of a performance ndex, such as one contanng state-control cross-weghtng terms. It s found that mult-agent systems wth agents havng consdered dynamcs on any drected graph, achevng consensus or synchronzaton, are nverse optmal wth respect to ths more general performance ndex. Inverse optmalty gves favorable propertes to a partally stablzng control law such as robustness and guaranteed gan margns, 4. 5

119 CHAPER 6 MULI-AGE SYSEMS WIH DISURBACES 6. Introducton he last two decades have wtnessed an ncreasng nterest n mult-agent network cooperatve systems, nspred by natural occurrence of flockng and formaton formng. hese systems are appled to formatons of spacecrafts, unmanned aeral vehcles, moble robots, dstrbuted sensor networks etc.,. Early work wth networked cooperatve systems n contnuous and dscrete tme s presented n,3,4,5,6,7. hese papers generally refer to consensus wthout a leader. We call ths the cooperatve regulator problem. here the fnal state of consensus depends on ntal condtons. By addng a leader that pns to a group of other agents one can obtan synchronzaton to a command trajectory usng a vrtual leader 5, also named pnnng control 8, 9. We call ths the cooperatve tracker problem. In the cooperatve tracker problem all the agents synchronze to the leader's reference trajectory. ecessary and suffcent condtons for synchronzaton are gven by the master stablty functon,, and the related concept of the synchronzng regon, n 9,,. For contnuous-tme systems, synchronzaton was guaranteed, 9,,3 usng local optmal state feedback derved from the contnuous tme algebrac Rccat equaton. It was shown that usng Rccat desgn of the feedback gan for each node guarantees an unbounded rght-half plane synchronzaton regon n the s-plane. hs allows for synchronzaton under mld condtons on the drected communcaton topology. Dual to the dstrbuted synchronzaton control problem s the dstrbuted estmaton problem, 3,3. Dstrbuted estmators are used when agents are only able to measure relatve nformaton n ther local neghborhoods. Output measurements are assumed and cooperatve dsturbance observers can be desgned for the mult-agent systems. Potental applcatons are dstrbuted observaton, dstrbuted dsturbance estmaton, sensor fuson, and dynamc output regulators for synchronzaton. Condtons for cooperatve observer convergence are shown n 6

120 3 to be related by a dualty concept to dstrbuted synchronzaton control condtons for systems on drected graphs. hs chapter s concerned wth the effects of dsturbances on mult-agent systems. Buldng on the classcal results on the exstence of Lyapunov functons for asymptotcally stable systems and ther use n assessng the effect the dsturbances exert on those systems, 46, t s possble to extend such reasonng to partally stable systems, n partcular those systems that reach consensus or synchronzaton. Wth Lyapunov functons for partal stablty one s able to ascertan the effect of dsturbances on the partal stablty of systems, and to derve condtons on those dsturbances that allow for asymptotc partal stablty or unform ultmate boundedness along the target set. Partal asymptotc stablty s n that sense robust to ths specfc class of dsturbances. Furthermore, wth the means to quantfy the effect of dsturbances one also gans the ablty to compensate for t by an approprate control law. In order to construct such a compensatng control dsturbances need to be known. However, the fact that dsturbances are usually naccessble to drect measurement ntroduces the need for dsturbance estmaton. he unfed approach to dsturbance estmaton of the leader's and agents' dsturbances n mult-agent systems s presented. Dsturbance estmaton and related compensaton s used to guarantee asymptotc state synchronzaton of agents that are nfluenced by dsturbances. he nteracton graph s drected and assumed to contan a drected spannng tree. For the needs of consensus and synchronzaton to a leader or control node we employ pnnng control. However, wth dsturbances present, the dstrbuted feedback pnnng control desgned for dsturbance free systems no longer guarantees synchronzaton. Robustness of the dstrbuted synchronzaton control for the nomnal system ndeed guarantees asymptotc cooperatve stablty or cooperatve ultmate unform boundedness n presence of some dsturbances, 46. hs property can be exploted n specal cases of heterogeneous and nonlnear agents. evertheless, n the general case of dsturbances one needs to compensate for ther effect n order to retan the qualtatve behavor of the nomnal system. For that purpose dsturbance 7

121 estmates are used. In ths chapter dsturbances are assumed to act on both the leader and the followng agents. herefore both the leader's and the agents' dsturbances need to be estmated and compensated. Leader's dsturbance estmate s obtaned by all agents va a dstrbuted estmator, whle the local agents' dsturbances can be estmated by local observers or, n a specal nstance, a dstrbuted observer. he structure of ths chapter s the followng; Secton 6. presents stablty defntons and propertes n a topologcal way whch s more general than usually presented n the lterature, 4, under the name of partal stablty. Converse Lyapunov result for partal stablty s also gven n ths secton for the sake of completeness snce nverse Lyapunov functons are used n later sectons. Coordnate transformatons that preserve the ntroduced stablty notons are also brefly addressed. Secton 6.3 gves the means to quanttatvely assess the effect dsturbances have on mult-agent systems and also allows characterzng the classes of dsturbances to whch the mult-agent system under consderaton s robust. Secton 6.4 ntroduces the leader's and the agents' dsturbance estmaton schemes. he results of Secton 6.4 are appled n Secton 6.5 and Secton 6.6. Secton 6.5 s concerned wth the case of a mult-agent system havng a leader drven by an nput. Secton 6.6 apples the same results to the mult-agent system comprsed of second-order double-ntegrator agents wth agents' dsturbances as well as the leader's nput present. Secton 6.7 and 6.8 dscuss how the robustness property of the nomnal system and dsturbance compensaton can be used to address cases of mult-agent systems wth heterogeneous and nonlnear agents. he example of a Lenard system was nvestgated n Secton 6.8 snce those systems are known to have an attractve lmt cycle, furnshng an example of a nontrval nvarant set for an solated agent. Secton 6.9 gves a numercal example based on the case presented n Secton 6.6 that justfes the proposed observer schemes and control laws. Concludng remarks are gven n Secton 6.. 8

122 6. Stablty Defntons and Propertes hs secton ntroduces partal stablty propertes and basc results that are used later n the chapter. Defntons gven here descrbe partal stablty n a general coordnate free way, n terms of neghborhoods of a target set. Defnton 6.: Let S be a manfold embedded n a Eucldean space X. A neghborhood D ( S) of S s an open set n X contanng the manfold S n ts nteror. Defnton 6.: Let S be a manfold embedded n a metrc space ( X, d ), and let D ( S) be a neghborhood of S. An ε -neghborhood of S D ( S) s defned as { ε} U ( S ) = x X d ( x, S ) <, where d( x, S): = nf d( x, y) s the dstance from x X to S as ε gven by the metrc d. y S ote that n the case of compact manfolds S, any neghborhood D ( S) contans some ε -neghborhood of S, but n the case of noncompact manfolds ths need not be true. For the needs of defnng stablty of noncompact manfolds S, one uses neghborhoods of S that contan some ε -neghborhood of S. We call such neghborhoods regular. Defnton 6.3: A manfold S s sad to be (Lyapunov) stable f there exsts a regular neghborhood D ( S) of S, such that for every ε -neghborhood U ( S ) contaned n t, there exsts a subneghborhood V ( S) satsfyng the property x() V( S) x( t) U ( S) t. If D ( S) can be taken as the entre space X, then the stablty s global. If S s Lyapunov stable and furthermore there exsts a neghborhood W ( S) of S satsfyng the property x() W ( S) d( x( t), S) as t, then S s asymptotcally stable. If d( x( t), S) we say that the trajectory x( t ) converges to S. ε ε 9

123 If S s Lyapunov stable and for every ε -neghborhood U ( S) of S, there exsts a subneghborhood V( S) U ( S) contanng a δ -neghborhood V ( S) then S s unformly ε stable. In ths case the stablty concluson can be phrased as ε >, δ > such that ( ( ) ) δ ( ( ) ) d x, S < d x t, S < ε t. If a manfold s unformly stable and asymptotcally stable so that some neghborhood W ( S) contans a δ -neghborhood W ( S) satsfyng the property x() W ( S) d ( x( t), S) as t, unformly on W ( S), then S s unformly δ asymptotcally stable. If a manfold s unformly asymptotcally stable and there exst constants K, σ > such that d( x( t), S) Kd( x(), S) e σt, for all x () n some δ -neghborhood W ( S ) then the partal δ δ δ ε δ stablty s exponental. he followng results address generally tme-varyng systems, where qualtatve propertes depend on ntal tme t ; x( t) = x. he exstence of a Lyapunov functon n case of unform-n-tme 3 asymptotc stablty of the orgn s a famlar result guaranteed under mld assumptons on the closed loop dynamcs x = f ( t, x), (38) 46, and for the case of unform asymptotc stablty of an nvarant manfold S the exstence follows from an analogous consderaton, usng the Massera's lemma (Lemma 6.). o present the proof, one needs the followng defntons and results. Results are presented n a slghtly more general form than needed here; n partcular, they are suted for tme-varyng systems. In ths context, unform convergence means ε >, > s. t. d( x( t), S) < ε t, where depends on ε and d ( x, S ), but not on x tself. 3 Unform-n-tme means that stablty and convergence do not depend on ntal tme t.

124 Proposton 6.. Gven a possbly noncompact nvarant manfold S and the dynamcal system (38), S s unformly stable f and only f there exsts a class K or K functon α and c > such that x() such that d( x(), S) < c. d ( x( t), S) α ( d ( x(), S)) he manfold S s unformly asymptotcally stable f there exst a class KL functon β and c > such that x() satsfyng d( x(), S) < c. d( x( t), S) β ( d( x( t ), S), t t ), (39) If the system s tme-nvarant one can smply wrte d ( x( t), S) β ( d ( x(), S), t) pckng ntal tme arbtrary. Proof follows by analogy wth results presented n 46, where the usual Eucldean norm,, s replaced wth a more general dstance, d(, S). he qualfcaton of stablty types n Proposton 6. refers to local stablty. For global stablty one needs K functons and the condton to hold c >. ote that the use of class KL functon guarantees unformty of asymptotc stablty n ths case for dfferent reason than presented n 46. here, unformty of convergence wth respect to the ntal tme was a consequence of the dependence of the KL class functon on t t, whle here t s because of the dependence on d( x, S ) and t t both, guaranteeng unformty of convergence to S wth respect to ntal tme as well as unformty of convergence along the manfold S,.e. dependng on d( x, S ), whch s the one we are manly nterested here. he proof of the Proposton 6. follows by analogy to the reasonng presented n 46.

125 We are motvated to prove the followng result concernng the exstence of a Lyapunov functon for unformly asymptotcally partally stable systems. heorem 6.. (Converse Lyapunov theorem) Suppose the system x = f ( t, x) has ts solutons defned for all future tmes t t, where t s the ntal tme, and s unformly asymptotcally stable wth respect to the manfold S U ( S). If the system satsfes the followng techncal condton d ( x, S ) f x ( t, x ) Ld ( x, S ), (4) on the neghborhood U ( S), then there exsts a Lyapunov functon defned there satsfyng α( d( x, S)) V ( t, x) α( d( x, S)), V ( t, x) α ( d( x, S)), 3 V ( t, x) α ( d( x, S)), 4 for α, α, α, α 3 4 class K functons. he followng lemma presents a useful techncal result. It serves n comparson systems nvolvng Lyapunov functons, as well as provdng an elegant proof of suffcency of the Lyapunov condton for partal stablty types as descrbed n Proposton 6.. Lemma 6.. Gven a dynamcal system, y = α ( y), wth y( t ) = y, where α s a locally Lpschtz class K functon defned on [, a ), there exsts a class KL functon σ defned on [, ) [, ) a such that y( t) = σ ( y, t t ). Lemma 6.. Massera's lemma, 46. Let g :[, ) (, ), be a contnuous strctly decreasng functon wth ( ) g t as t. Let [ ) functon. hen there exsts a functon G( t ) such that G( t ) and G '( t ) are class K functons defned on [, ). h :, (, ), be a contnuous nondecreasng

126 For every contnuous u( t ), such that u( t) g( t), t, there exst postve constants, K, K, ndependent of u, such that G( u( t)) dt K and G '( u( t)) h( t) dt K. In the followng lemma the mplct assumpton s made that the manfold S s embedded n the Eucldean space, wth ts usual dfferental and vector space structure. he neghborhood U ( S) under consderaton s thus a subset of the Eucldean space nhertng all the dfferental structure. Lemma 6.3. he gradent of the dstance functon d x ( x, S ) s unformly bounded. Proof: Assume one chooses two arbtrary ponts x, y U ( S), and that the nfmum dstance from the manfold S s actually attaned at ponts x, y S such that d( x, S) : = nf d( x, z) = d( x, x ), z S d( y, S) : = nf d( y, z) = d( y, y ). z S hen d( y, S) d( y, x ) d( y, x) + d( x, x ) = d( y, x) + d( x, S), wherefrom t follows that d( y, S) d( x, S) d( y, x). Exchangng x and y one obtans d( x, y) d( x, S) d( y, S) d( x, y), therefore d( x, S) d( y, S) d( x, y), provng Lpschtz contnuty of the functon d(, S). hs also means that each drectonal dervatve satsfes d( x + tv, S) d( x, S) d( x + tv, x) lm =, t tv tv mplyng boundedness of the gradent, d x ( x, S ), on U ( S). 3

127 Remark 6.: Here the vector space structure of U ( S) s mplctly used. Otherwse the term x + tv makes no sense. Also, the assumpton that the nfmum dstance s attaned for some pont on the manfold S can be made redundant f one nvokes the property of nfmum, that t can be approached arbtrarly closely. hen one can use an approxmatng pont on the manfold S, ntroducng arbtrarly small devatons from the actual nfmum. However, snce the devatons are allowed to be arbtrarly small, the derved nequaltes hold just as well. Proof of heorem 6.. Let us denote the soluton to the dynamcs x = f ( t, x) wth ntal state x at tme t as x( t) ( t, t, x) = φ. Snce d x t S β d x t S t t ( ( ), ) ( ( ( ), ), ) one has d( φ( t, t, x), S) β( d( x, S), t t ). Let us make use of the followng three techncal condtons he soluton x( t) = φ( t, t, x) can be extended t t, d x ( x, S ) < K, x U ( S), d ( x, S ) f x ( t, x ) Ld ( x, S ), x U ( S). d Snce d ( x, S ) = (, ) (, ) x d x S f t x one has dt d d ( x, S ) = x d ( x, S ) f ( t, x ) Ld ( x, S ), dt d Ld( x, S) d( x, S) Ld( x, S) dt d( x( t ), S) e d( x, S) d( x( t ), S) e L ( t t ) L( t t ) establshng a dfferent bound on d( x, S ). Let the Lyapunov functon be defned as t+ V ( t, x) = G( d( φ( τ, t, x), S)) dτ, wth functon G whose exstence s guaranteed by Massera's lemma. hen t 4

128 t+ t+ t+ L( τ t) ( (, ) ) ( ( (,, ), )) ( ( (, ), ) G d x S e dτ G d φ τ t x S dτ G β d x S τ t dτ, t t t t+ L( τ t) Ls ( (, )) : = ( (, ) ) τ = ( (, ) ) t, α d x S G d x S e d G d x S e ds α ( d( x, S)) : = G( β ( d( x, S), τ t)) dτ = G( β ( d( x, S), s) ds t+. t Further, one makes use of the assumpton that the orbt can be extended to all future tmes. For compact manfolds, local Lpschtz contnuty on a compact neghborhood guarantees extendblty of solutons to all future tmes, but for neghborhoods of general noncompact manfolds ths condton needs to be mposed snce an escape to nfnty n fnte tme along the noncompact manfold remans a possblty. he tme dervatve of the Lyapunov functon equals d V ( t, x ) = V ( t, x ) + V ( t, x ) f ( t, x ) dt t t+ [ ] = G( d( φ( t +, t, x), S)) G( d( x, S)) + G'( d( φ( τ, t, x), S) d( φ, S) φ ( τ, t, x) + φ ( τ, t, x) f ( t, x) dτ. Owng to the dentty φ ( τ, t, x) + φ ( τ, t, x) f ( t, x), τ t, one has t x t φ t x d V ( t, x ) = G ( d ( φ( t +, t, x ), S ) G ( d ( x, S )) G ( β ( d ( x, S ), )) G ( d ( x, S )). dt Snce, by assumpton on the exstence of soluton for all future tmes, can be chosen arbtrarly large, so that as β(, ) when, one wll have, by contnuty, that G β(, ). Hence d V ( t, x ) G ( d ( x, S )) : = α3( d ( x, S )). dt One also has the bound on the gradent 5

129 t + xv ( t, x) = G '( d( φ, S)) φd( φ, S) φxdτ, t+ whch mples xv ( t, x) G '( d( φ, S)) φd( φ, S) φx dτ. t t If f ( t, x ) < M, whch s satsfed n partcular by lnear systems, then M ( t) x φ x < e τ, furnshng the bound t+ M ( τ t) xv ( t, x) G '( d(, S)) φd(, S) e d t φ φ τ t+ M ( t τ ) G '( d( φ, S)) Ke d t t+ M ( τ t) G '( ( d( x, S), t)) Ke d t t+ Ms G '( β ( d( x, S), s)) Ke ds : = α ( d( x, S)). t τ β τ τ 4 he exstence of a class K functon α 4 s guaranteed by Massera's lemma (Lemma 6.). In case of tme-nvarant dynamcs the soluton satsfes x( t) = φ( t, t, x) = φ ( t t, x), wth no explct dependence on the ntal tme t. hs means that the expresson for the Lyapunov functon becomes V ( t, x) = G( d( φ( τ, t, x), S)) dτ = G( d( φ ( τ t, x), S)) dτ V ( x) = G( d( φ ( s, x), S)) ds, t+ t+ t wth no explct tme dependence. Snce solutons are assumed to exst for all future tmes, one can use the above expresson wth, yeldng t = V ( x) G( d( φ ( s, x), S)) ds. 6

130 he above ntegral exsts by Massera's lemma (Lemma 6.). he specal case of unform asymptotc stablty, n whch one has exponental stablty, t t where the KL class functon β( d( x, S), t t ) s the exponental, d( φ( t, t, x), S) Kd( x, S) e σ, ( ) proves to be suffcently general for our purposes. In that case, a Lyapunov functon can be chosen as t+ (, ) ( φ( τ,, )) V t x = d t x dτ. t he class K functons α, α, α 3 become quadratc functons of d ( x, S ), and α 4 s a lnear functon of d ( x, S ), 46. cd ( x, S) V ( t, x) cd ( x, S), V t x c d x S (, ) 3 (, ), V ( t, x) c d( x, S). 4 he followng developments necesstate a precse defnton of the neghborhoods under consderaton. Defnton 6.4: A constraned regular neghborhood of S s a regular neghborhood of S contaned n some ε -neghborhood of S. herefore, constraned regular neghborhoods of S are those that contan some ε - neghborhood and are contaned n some ε -neghborhood of S. ote that ε -neghborhoods themselves are constraned regular neghborhoods by defnton. hese are precsely the neghborhoods relevant for our sense of stablty. One should bear n mnd that the stablty propertes of Defnton 6.3 are not nvarant wth respect to all homeomorphsms or even dffeomorphsms. One can have a homeomorphsm mappng a constraned regular neghborhood onto one that s not constraned x regular. An example s easly constructed; the dffeomorphsm ( x, y) ( x ', y ') = ( x, ye ), 7

131 preserves the x-axs and maps a constraned regular neghborhood of the x-axs to a neghborhood of the x'-axs that s not constraned regular. he dynamcal system x = y = y, whch s exponentally stable wth respect to the x-axs, s transformed to x ' = y ' = y ', whch s clearly not stable wth respect to the x'-axs, although the x'-axs remans nvarant. Lnear coordnate transformatons, beng unformly contnuous, wth unformly contnuous nverse, ndeed preserve the stablty notons of Defnton 6.3. hs s elaborated n the followng lemma. Lemma 6.4. Let a mappng h : U ( S) X, defned on a neghborhood, U ( S) X, of S, be a homeomorpsm onto ts mage, V( h( S)) = h( U ( S)). he property of neghborhoods of S beng constraned regular, and subsequently the sense of dstance from S, are preserved under the mappng h f there exst class K functons α, β such that α ( d( x, S)) d( h( x), h( S)) β ( d( x, S)). Such coordnate transformatons also preserve the convergence of the trajectory to the manfold S. Lkewse, any Lyapunov functon, V ( x ), for unform stablty wth respect to the manfold S retans ts unformty propertes n the transformed coordnates. Proof: It suffces to show that such a mappng maps constraned regular neghborhoods of S onto constraned regular neghborhoods of the mage of S. he same needs to hold for the nverse mappng. If U ( S) satsfes d( x, S) c, x U ( S), then d( h( x), h( S)) β( c) x U ( S). Hence y h( U ( S)), d( y, h( S)) β ( c) = c. If a neghborhood U ( S) of S s regular,.e. t 8

132 contans some ε -neghborhood U ( S), then the exteror of U ( S) n U ( S), U( S) \ U ( S), gets ε mapped to the exteror of U ( h( S)) ( ) n h( U ( S)). So, the mage h( U ( S)) contans α ε U ( h( S)) α ( ε ) ; therefore t s a regular neghborhood. If the same property does not hold for the nverse mappng, one would have h volatng one of ts bounds mposed by functons α, β. Also, f d( x( t), S) as t, one has d( h( x( t)), h( S)) β ( d( x( t), S) as t, so that the property of the convergence of trajectores s preserved under the mappng h. As for the Lyapunov functon V ( x ), let t satsfy the condton ε ε α ( d( x, S)) V ( x) α ( d( x, S)) for some class K functons, α α. Let x h( y) = be the mappng satsfyng the condton of the Lemma, α ( d( y, S)) d( h( y), h( S)) β ( d( y, S)). hen one fnds that α ( α( d( y, h ( S))) α ( d( h( y), S)) V ( h( y)) α ( d( h( y), S)) α ( β ( d( y, h ( S))). Defnng class K functons α = α α, α = α β and V = V h one has α ( d( y, h ( S))) V ( y) α ( d( y, h ( S))), whch s the unformty condton n the transformed coordnates, y, owng to the fact that the composton of class K functons s agan a class K functon, 46. A smlar argument apples to the condton on the dervatve of the Lyapunov functon, completng the proof. Remark 6.: Unformly contnuous homeomorphsms havng unformly contnuous nverse satsfy the condton of Lemma 6.4. Unform contnuty of the mappng guarantees the upper bound, and unform contnuty of the nverse guarantees the lower bound. Lnear transformatons, n partcular, belong to ths class. hs requrement should be contrasted wth 9

133 the smpler contnuty condton that s a defnng property of a homeomorphsm. Homeomorphsms preserve open sets, whle homeomorphsms satsfyng the more strngent condton of Lemma 6.4 preserve constraned regular neghborhoods, descrbng the presented notons of stablty. hese are preserved for compact manfolds and ther precompact neghborhoods, by vrtue of compactness, under usual homeomorphsms, but for general noncompact manfolds the addtonal condton of Lemma 6.4 s needed. herefore, when usng coordnate transformatons on dynamcal systems exhbtng stablty wth respect to a noncompact manfold S, one should exercse care snce homeomorphsms, and even dffeomorphsms, do not preserve the stablty propertes unless they satsfy the more strngent condton of Lemma 6.4. hs, n partcular, apples to feedback lnearzaton methods, when transformatons of sngle-agent state-space comprse the transformaton of the total state-space X, and the target manfold S embedded n t. More subtly, one can have topologcally equvalent systems that do not exhbt the same stablty propertes f the target set s noncompact. 6.3 Assessng the Effects of Dsturbances on Mult-agent Systems Usng Lyapunov Functons hs secton ntroduces the dynamcs of the mult-agent system wth an optonal presence of the leader. he agents are assumed to be n control affne form, and of the same order. Here the Lyapunov functons are used to quantfy the effect that dsturbances have on the unformly asymptotcally stable closed-loop undsturbed system. Specfed to lnear tmenvarant systems, bounds on dsturbances are expressed n a more precse way, usng quadratc Lyapunov functons Mult-agent System Dynamcs wth Dsturbances Consder a mult-agent system wth an optonal presence of a leader. Agents are descrbed as nodes of a drected graph endowed wth dynamcs. Edges of the graph represent the communcaton structure,.e. denote whch agents' states are avalable for the purpose of

134 the cooperatve feedback to a gven agent. It s assumed that the full state of neghborng agents s avalable for feedback purposes. Assume that the leader node dynamcs s x = f ( x ) + ξ, (4) and the dentcal agents have dynamcs n the control affne form wth, n x = f ( x ) + g( x ) u + ξ, (4) x x R and, p u u R. For the purposes of ths chapter, functons f, g descrbe a nomnal system. Sgnals ξ and ξ are the agent's and leader's dsturbances. he nature of the dsturbances can be varous. Dsturbance terms can even come from unmodelled dynamcs, allowng dfferent drft dynamcs of each agent, as long as the dynamcal systems are of the same order. hs furnshes an nstance of a state dependent dsturbance ξ ( x ). Defnton 6.5: he dstrbuted consensus or synchronzaton problem s to fnd dstrbuted feedback controls u ( x ) for agents that guarantee lm x ( t) xj ( t) =, j when there s no leader, and u ( δ ), where δ = x x, that guarantee lm x ( t) x ( t) = when the t t leader x s present. We call the former cooperatve regulator problem and the latter cooperatve tracker problem. he target sets are the consensus manfold S : x = x, (, j) n j the total state-space n X =R and S :, δ =, subset of the ( δ, x ) space X ( + ) n =R. Assumpton 6.. One has found the cooperatve feedback control law u( x ) that guarantees asymptotcally stable consensus for the nomnal system,.e. solves the cooperatve regulator problem. Alternatvely, one has the cooperatve feedback control law u( δ ) that guarantees asymptotcally stable synchronzaton of the nomnal system,.e. solves the cooperatve tracker problem for the nomnal system. Asymptotc stablty s assumed unform. he closed loop nomnal systems are gven as

135 x = F ( ) : ( ) ( ) ( ) cl x = F x + G x u x, (43) δ = F ( δ, x ) : = F( x) F( x ) + G( x) u( δ ), x cl = f ( x ), (44) where F( x) = f ( x ) f ( x ), G( x) = dag ( g( x ),..., g( x )), and x n (44) s replaced by δ + x. Under Assumpton 6., by the converse Lyapunov theorem (heorem 6.), there exst Lyapunov functon for stablty of the consensus manfold, V ( x ), and a Lyapunov functon δ x for the error sgnals V (, ) δ = x x, for the closed loop nomnal systems (43), (44), respectvely. Actually, the latter case s a classc case of partal stablty, 4, n whch δ (, ) measures the dstance from δ = subspace of the total δ x space. ote that a specal case of the Lyapunov functon, V ( δ, x ), for partal stablty, havng the form V ( δ ), suffces n some nstances. he effects of dsturbances can be assessed usng those Lyapunov functons. Just as n the classcal case, 46, one has asymptotc stablty wth respect to the consensus manfold or unformly ultmately bounded consensus 4, dependng on the knd of bounds that the dsturbances satsfy. o elaborate further on dfferent types of dsturbance bounds, let us start wth the followng result formulated as a theorem. he Lyapunov functons under consderaton are allowed to depend explctly on tme, for the sake of generalty and because ths changes nothng conceptually, as long as they satsfy the same tme ndependent unformty bounds. hs generalzaton allows one to treat tme-varyng agents as well. heorem 6.. Let the closed loop nomnal systems (43), (44), satsfy the Assumpton 6.. Let the dsturbances satsfy the unform growth bound ξ ( x) α ( d( x, S)), (45) 5 for the cooperatve regulator problem, and 4 he unformly ultmately bounded consensus means that ( ε > ) ( > ) s. t. d( x( t), S) < ε t >.

136 ξ ( x) ξ ( x ) α ( δ ), ξ < M, (46) 5 for the cooperatve tracker problem. hen, for a proper choce of class K functon α 5, one has asymptotc stablty n presence of dsturbances. If the dsturbances and the Lyapunov functons satsfy the unform bound for some M, M V ( t, x) ξ M, (47) for the cooperatve regulator problem, and δv ( t, δ, x )( ξ ξ ) M, x V ( t, δ, x ) ξ < M (48) for the cooperatve tracker problem, then the resultng convergence to consensus s unformly ultmately bounded. Proof: Wth the cooperatve feedback control, u( x ), satsfyng Assumpton 6. for the nomnal system (43), the closed loop nomnal system x = F( x) + G( x) u( x) = F ( x), cl s unformly asymptotcally stable wth respect to the consensus manfold S. herefore, by heorem 6., there exsts a Lyapunov functon, V ( t, x ), for unform asymptotc stablty wth respect to the consensus manfold n S R. Wth the cooperatve feedback control, u( δ ), satsfyng Assumpton 6. for the nomnal system (44), the closed-loop nomnal system δ = F( x) F( x ) + G( x) u( δ ) = F ( δ, x ), x = f ( x ), cl s unformly asymptotcally stable to the δ = subspace. herefore, by heorem 6., there exsts a Lyapunov functon, V ( t, δ, x ), for unform asymptotc stablty wth respect to δ = subspace. hose functons satsfy the followng 3

137 V( t, x) ; V( t, x) = x S α ( d( x, S)) V( t, x) α ( d( x, S)) d V( t, x) = V( t, x) + V ( t, x) Fcl ( x) α 3( d( x, S)) dt t for the cooperatve regulator and V( t, δ, x ) > ; V( t, δ, x ) = δ = α ( δ ) V( t, δ, x ) α ( δ ) d V ( t, δ, x ) = V ( t, δ, x ) + V ( t, δ, x ) F ( δ, x ) V ( t, δ, x ) f ( x ) α ( δ ) dt t δ cl + x 3, for the cooperatve tracker. he α, α, α 3 are some class K functons. In the presence of dsturbances the global system (43) takes the form x = F ( ) cl x + ξ, (49) and the error system (44) equals δ = F ( δ, x ) + ξ ξ, x cl = f ( x ) + ξ. (5) In that case Lyapunov functons for nomnal cooperatve regulator and tracker systems, respectvely, satsfy d V ( t, x ) α3( d ( x, S )) + V ( t, x ) ξ, (5) dt and d V ( t, δ, x ) α3( δ ) + δ V ( t, δ, x )( ξ ξ) + x (,, V t δ x ) ξ. (5) dt If there exsts a class K, or globally K, functon α ( d( x, S)) 3 such that α ( d( x, S)) + V ( x, t) ξ α ( d( x, S)), (53) 3 3 4

138 then (5) mples that the consensus s unformly asymptotcally acheved n spte of the dsturbances. he same apples to the asymptotc stablty for the cooperatve tracker problem (5) f there exsts α ( δ ) 3 such that α ( δ ) + V ( t δ, δ, x )( ξ ξ ) + V ( t, δ, x ) ξ α ( δ ). (54) 3 x 3 Gradent of the Lyapunov functon satsfes the growth bound V ( t, x) α ( d( x, S)), n the 4 case of the cooperatve regulator problem, or the bound δv ( t, δ, x ) α ( δ ), and 4 x V ( t, δ, x ) α ( δ ), n the case of the cooperatve tracker problem. If the dsturbances 4 satsfy bounds ξ ( x) α ( d( x, S)), or ξ( x) ξ ( x ) α ( δ ) and ξ < M 5 5, t s possble that for a proper choce of class K functons α3, α4, α 5 and bound M one can guarantee unform asymptotc stablty n presence of dsturbances, 46. In that case one has a sought α 3, class K functon, such that d V ( x, t ) α ( (, )) ( (, )) ( (, )) : ( (, )) 3 d x S + α4 d x S α5 d x S = α3 d x S, dt for the cooperatve regulator and d V ( δ, t, x ) α ( δ ) + α ( δ )( α ( δ ) + M ) : = α ( δ ), dt for the cooperatve tracker. In the case of the unform bounds (47), (48), the resultng convergence to consensus s unformly ultmately bounded, snce for suffcently large d ( x, S ) or δ the dervatve of a Lyapunov functon s negatve, 46. he numercal value of the ultmate bound depends on the choce of the Lyapunov functon. hs concludes the proof. Dsturbances comng from unmodelled dynamcs are lkely to satsfy the former case, and generally unknown dsturbances that come from the envronment are more lkely to ft nto the latter case. If the nomnal cooperatve tracker admts a Lyapunov functon V ( t, δ, x ) havng 5

139 a specal form, V ( t, δ ), the robustness condtons of heorem 6. smplfy. hs shall be the case n the followng subsecton Specfcaton to Lnear me-nvarant Systems Havng agents and the leader n a form of lnear tme-nvarant systems, one can derve more specfc bounds on the dsturbances and the regon of attracton, usng quadratc Lyapunov functons. he dynamcs of the leader and dentcal agents s gven as x = Ax + ξ, (55) x = Ax + Bu + ξ, (56) wth, n x ; x R,, p u ; u R. For the purposes of ths chapter matrces A, B descrbe the nomnal system. Sgnals ξ and ξ are the agent and leader dsturbances. he global systems for cooperatve regulator and tracker problems wth dsturbances are respectvely x = ( I A) x + ( I B) u + ξ, (57) and defnng the global dsagreement vector δ = δ δ, where δ = x x one has the system δ = ( I A) δ + ( I B) u + ξ ξ. (58) = Here ξ ξ ξ. Defne the local neghborhood error as e = e ( x x ) + g ( x x ). (59) j j j he lnear cooperatve feedback control s chosen as u = cke. (6) In global form ths s u = cl Kx or u = c( L + G) Kδ, dependng on whether there are nonzero pnnng terms or not. he closed loop nomnal systems are, (6) x = A x + ξ cl 6

140 . (6) δ = δ + ξ ξ A cl he closed-loop system matrces A = I A cl BK and A = I A c( L + G) BK cl cl need to be asymptotcally stable wth respect to the consensus manfold, or wth respect to δ = subspace, respectvely. ote that, because of the lnearty of the drft dynamcs, x does not explctly appear n (6). hs, n partcular, motvates our choce of u( δ ) for the cooperatve tracker feedback control nstead of a more general control law u( δ, x ), as well as our choce of the Lyapunov functon V ( δ ), nstead of V ( δ, x ). hese choces reduce the lnear cooperatve tracker problem to a classcal problem of asymptotc stablty of the orgn. K Acl A cl One possble choce of the local feedback gan that stablzes and s gven by the followng lemma. Lemma 6.5. Assume that a graph has a spannng tree wth at least one pnnng gan nonzero, connectng nto the root node when pnnng s present. Let the par ( A, B) be stablzable. Choose the local feedback gan K as where P K = R B P, (63) s the soluton to the algebrac Rccat equaton A P + PA + Q PBR B P =. (64) hen the closed loop system matrx A cl s asymptotcally, exponentally, stable to the consensus manfold f the couplng gan s suffcently great so that c mn Re j λ j >, (65) λ j > L A cl where are the postve egenvalues of the matrx, and the closed loop matrx s asymptotcally, exponentally, stable f the couplng gan s suffcently great so that 7

141 λ L + G where are the egenvalues of the matrx. j c, (66) mn Reλ j j Proof s gven n 3. Wth ths result, one can guarantee that the lnear mult-agent systems, (55), (56), satsfy Assumpton 6.. he followng conclusons apply when dsturbances are assumed to act on the system. Corollary 6.. If dsturbances, ξ, satsfy the unform growth bound, d( ξ, S) < Cd( x, S) 5, gven that a growth constant C s suffcently small, one has unform asymptotc stablty of consensus. If dsturbances satsfy the bound d( ξ, S) < M, one has unformly ultmately bounded convergence to consensus. For the cooperatve tracker, the dsturbance bound ξ ξ mples asymptotc stablty, gven that a growth constant < C δ C s suffcently small. On the other hand, the dsturbance bound ξ ξ < M mples unform ultmate boundedness of the error dynamcs. Proof: he relevant Lyapunov functons for the lnear systems (6), (6), satsfyng Assumpton 6., exst n quadratc form as ( ) V x = x Px and V ( δ ) = δ Pδ > wth matrces P, P >, where ker P ; the consensus manfold. herefore one has a = S Lyapunov functon for the unform asymptotc stablty satsfyng V ( x) = x Q x σ ( Q ) d( x, S) > mn and the Lyapunov functon satsfyng V ( δ) = δ Q δ <. In presence of dsturbances one fnds the followng denttes 5 hs s a slght abuse of notaton. he dsturbance ξ, as a part of the vector feld defnng dynamcs of the system, s not an element of the state space X, but of the tangent space vector spaces, R, ths ought to cause no confuson. 8 X, and d( ξ, S) s a dstance from the consensus manfold n that tangent space. However for lnear systems on Eucldean n x

142 V ( x) = x Q x + x Pξ (67) and where Q, wth ker Q, and. = ker P Q > V ( δ ) = δ Q δ + δ P ( ξ ξ ), (68) Owng to the fact that both quadratc forms n (67), x Px and x Q, measure the dstance x from the consensus manfold, they are equvalent n the sense of dstance norms,.e. they can be bounded by above and below by a scaled dstance from the consensus manfold d( x, S). Hence, for V one can state V x σ ( ) ( Q) d( x, S) + ( mn ) ( > σ ( Q) d( x, S) + P x Pξ > mn σ + ( Q) d( x, S) Kd( x, S) d( ξ, S) > mn ( σ ( Q) d( x, S) Kd( ξ, S)) d( x, S), > mn P x Pξ ) (69) for some real constant K >. If there exsts a dsturbance bound d( ξ, S) < Cd( x, S) or d( ξ, S) < M one wll have asymptotc convergence to consensus manfold S n the former case f a growth constant case, 46. C s suffcently small, or unform ultmate boundedness n the latter For cooperatve tracker, postve defnteness of and Q allow for an applcaton of P the classcal results on dsturbances, 46. Accordng to those results one has, respectvely, asymptotc stablty and unform ultmate boundedness wth respect to the orgn of the synchronzaton error system. ote that a more general bound on d( ξ, S), by a K class functon of d( x, S) that satsfes the lnear growth bound locally, also suffces. 9

143 In many realstc cases of dsturbances the robustness alone, stemmng naturally from the asymptotc or the exponental cooperatve stablty of the dsturbance free systems, does not suffce. In those nstances t becomes advantageous to measure or estmate the dsturbances actng on the leader and the followng agents, so that those could be compensated for, even f only n part. Dsturbance estmaton schemes for the leader's and agents' dsturbances are presented n the next secton. 6.4 Dsturbance Estmators In ths secton we ntroduce the dsturbance estmaton of the leader's and the agents' dsturbance. he mportant fact s that n cooperatve tracker problem, n addton to ts own ξ ξ dsturbance sgnal, each agent also needs to know the leader's dsturbance. In realty those need to be replaced wth ther estmates. hs ntroduces the need for estmators of the leader's and the agents' dsturbances. In the followng subsectons, local and dstrbuted observers are ntroduced for the purpose of estmatng local dsturbances. Local observers are a natural choce for estmatng the agents' dsturbances f measurements of those dsturbances are avalable. In the specal case of all agents havng the same dsturbance generator one can estmate the local dsturbances by a dstrbuted observer. In fact f only relatve dsturbance measurements are avalable to all agents, except to a few that know the absolute reference value, one needs to use the dstrbuted observer. Leader's dsturbance observer naturally has a dstrbuted form Estmatng Dsturbances on the Agents hs and the followng subsecton are concerned wth estmatng the agents' dsturbances. Apart from the leader's dsturbance estmate ξˆ each agent needs to know only ts own dsturbance estmate ξˆ, not at all ts neghbor's dsturbances. hs s a consequence of the fact that dsturbances actng on agents are assumed ndependent, so ξ acts solely on agent. 3

144 Assume that the dsturbance actng on agent dsturbance sgnal generator ξ = Γ ξ ς = Fξ s modelled by the followng lnear (7) ξ he dsturbance actng on the agent cannot be measured drectly,but a related quantty, the output of the dsturbance generator, s accessble to measurements. hus one uses the pertanng dsturbance generator to desgn a local estmator. ς ˆ ξ = Γ ˆ ξ + L ( ς ˆ ς ) ˆ ς = F ˆ ξ (7) If the matrx Γ L F s Hurwtz the estmate error ξ ˆ ξ wll converge to zero. hs s a classcal local Luenberger estmator. If and only f the par ( Γ, F ) s detectable one can desgn the observer gan that guarantees stablty of Γ L F,.e. convergence of estmate to the L true value of the dsturbance Case of Identcal Dsturbance Generators for all Agents In the specal case when the dsturbance generators for all agents are the same, a dstrbuted dsturbance observer can make all dsturbance estmates converge to the true values of the dsturbance sgnals, as long as there s a pnned reference value of to some agents. hs amounts to some agents knowng the absolute zero reference, whle all others rely on relatve nformaton only. Hence, agents estmate ther own dsturbances, but the estmaton s performed n a dstrbuted fashon. herefore, the entre mult-agent system estmates the total dsturbance vector ξ. hs s made possble n such a form precsely because all the dsturbance generators are assumed to be the same ξ = Γξ ς = Fξ =,,...,. (7) 3

145 he dstrbuted dsturbance estmator, 3, has the form ˆ ξ = Γ ˆ ξ c L e ( ς ς ) g ς j j j ς = ς ˆ ς ˆ ς = F ˆ ξ, (73) and guarantees estmate convergence to the true dsturbance value f the observer system matrx I Γ c ( L+ G) L F (74) s Hurwtz. Lemma 6.6. Assume that the graph has a spannng tree wth at least one pnnng gan nonzero, connectng nto the root node, and that the par ( Γ, F) s detectable. Choose the local observer gan L as where P L = PF R, (75) s the soluton to the observer algebrac Rccat equaton Γ P + PΓ + Q PF R FP =. (76) hen the observer system matrx (74) s asymptotcally, exponentally, stable f the couplng gan s suffcently great λ L + G where are the egenvalues of the matrx. j Proof s gven n 3. c, (77) mn Reλ j j Local estmaton remans an opton n ths case as n 9, wth the same choce of the observer gan (75),(76), guaranteeng convergence of local observer (7). It should be noted that an assumpton on dentcal dsturbance generators for dstnct agents mght be too naïve for most real cases. 3

146 6.4.3 Estmatng Dsturbance on the Leader wth Known Leader's Dsturbance Generator In ths subsecton the dstrbuted leader's dsturbance observers are ntroduced. All the followng agents need to have a leader's dsturbance estmator ξˆ. In case all agents know the leader's dsturbance generatng system ξ = Γ ξ ς = F ξ, (78) f one pns the measurement ς nto a selected few nodes then the followng synchronzaton type dstrbuted estmator archtecture s avalable. he dstrbuted leader's dsturbance estmators take the classcal cooperatve tracker form wth local neghborhood output dsagreement feedback ˆ ξ = Γ ˆ ξ + c L e ( ˆ ς ˆ ς ) + g ( ς ˆ ς ) 3 j j j ˆ ξ = Γ ˆ ξ + c L F e ( ˆ ξ ˆ ξ ) + g ( ξ ˆ ξ ), 3 j j j (79) where ˆ ˆ = Fξ ς. (8) If the observer system matrx I Γ c ( L + G) L F 3 s Hurwtz then all local ˆ ξ ξ estmates converge to the true value of the leader's dsturbance asymptotcally. he desgn of c, L guaranteeng stablty of the observer system matrx s detaled n Lemma Estmatng Dsturbance on the Leader wthout the Leader's Dsturbance Generator hs subsecton ntroduces the dstrbuted leader's dsturbance observers wthout havng the leader's dsturbance generatng system. If the leader's dsturbance generator s not known but there exsts a fnte k such that ξ = ( k ) dentcally then a k-th order pnned consensus algorthm for the cooperatve observer, 33

147 k j ˆ( k ) d ˆ ˆ ˆ ξ = c j el ( ξl ξ ) + g ( ξ ξ ), j j= dt l (8) can guarantee convergence of all leader's dsturbance estmates to the true value. ote that only the pnned nodes can measure the leader's dsturbance ξ drectly. Others rely on ther local neghborhood estmates ξˆ j, or more precsely on local neghborhood relatve estmate p R p dsagreement. All dsturbances and ther estmates are assumed to be elements of, or m. he coeffcents are chosen to guarantee stablty,.e. so that δ = ˆ ξ ξ converges ck ξ to zero asymptotcally. Indeed, by canoncally assgnng state varables ( k ) y, one fnds the system n block controllable = δξ, y = δ, ξ y k δ = ξ y = y y k canoncal form I p I p d. (8) y = y dt I p c ( L G) I p c ( L G) I p ck ( L G) I p Lemma 6.7. he characterstc polynomal of the block controller canoncal form matrx (8) equals k k k P( s) = det( si M ) = det( s I + ( c s + c s + + c )( L+ G) I ), (83) kp k p k k p = n Proof: Let the bases of the nducton be m = si p I p det( sip M ) = det c ( L G) I p si p c ( L G) I p = det( si p )det( si p + c ( L + G) I p + c( L + G) I p ) s = det( s I p + ( cs + c)( L + G) I p ), verfyng the asserton for the bass. Let M m be the block matrx of the form 34

148 M m I p I p = I p c( L + G) I p c ( L + G) I p cm ( L + G) I p m m Assume for m that det( si M ) = det( s I + ( c s + + c )( L + G) I ), then one has mp m p m p the followng dentty for m + M m+ I p I p = I p c ( L + G) I p c ( L + G) I p cm ( L + G) I p cm ( L + G) I p and si p I p det( si( m ) p M m ) simp M + + = m c( L + G) I p M c,, cm where the coeffcents n the last block row of go from. m det( si( m ) p M m ) det si p det(( simp M ) + + = m I p ) s c ( L + G) I p si p I p si p p = s det si p I p ( c + c )( L + G) I p c( L + G) I p si p + cm ( L + G) I p s By nducton assumpton, ths equals 35

149 = s s I + c s + + c + c L + G I s = p m m det( p ( m )( ) p ) m+ m det( s I p ( cms cs c )( L G) I p ) completng the nducton proof. Remark 6.3: It should be noted that for hgher order k the proposed algorthm for correctons ξˆ becomes ncreasngly complcated. ecessary order s determned by the constrant on the ξ ξ dsturbance sgnal. If s modeled by e.g. splne functons (of preferably lower order) then such an algorthm suffces. Also, f the hgher order dervatves are not dentcally zero but can be consdered neglgble, or otherwse small n some sense, compared to a fnte number of lower order ones, the applcaton of such an observer s stll justfed owng to the fact that those small terms ntroduce a comparatvely small perturbaton of the asymptotcally stable system, resultng n a small steady state error. he choce of the coeffcents c j has to be made such that the asymptotc stablty s guaranteed. hs consttutes the hgher order observer desgn problem. In case stablty s ( ) ( ) guaranteed one has lm y =, meanng as t, = m for all m =,, k. Most t mportantly the steady state error δ ξ ˆ m ξ ξ converges to zero, whch means the convergence of the estmate to the true value of the dsturbance. One way of choosng the coeffcents c j n relaton to the graph matrx L + G egenvalues s detaled n the followng theorem. heorem 6.3. Let A, B C C be the sngle nput k-th order controller canoncal form matrces wth all characterstc polynomal coeffcents equal to zero A C =, B. (84) C = 36

150 he optmal feedback gan where P [ ] K = R BC P = c ck s the soluton of the algebrac Rccat equaton, (85) A P + PA + Q PB R B P =, (86) C C C C c gves coeffcents that guarantee stablty f all the egenvalues λ of the matrx L + G are j n the complex gan margn regon, 9, for the Rccat feedback (85). Proof: In order to smplfy the determnant of a matrx polynomal k k k P( s) = det( si M) = det( s I + ( c s + c s + + c )( L+ G) I ), kp p k k p one can apply the lnear transformaton reducng ( L + G) I p to the upper trangular form,.e. f ( L + G) = Λ, wth Λ upper trangular, then I s the desred transformaton for p ( L + G) I p. Applyng such a transformaton on the matrx polynomal j s I + ( c s + c s + + c )( L + G) I k k k p k k p does not change the determnant and yelds the smplfed form ( s I + ( c s + c s + + c )( L + G) I ) = s I + ( c s + c s + + c ) Λ I k k k k k k p k k p p k k p = ( s I + ( c s + c s + + c ) Λ) I k k k k k p whch s tself n an upper trangular form. he determnant s now smply the product of dagonal elements havng the followng form k k k Pj ( s) = s + ( ck s + ck s + + c) λ j. (87) = s + c λ s + c λ s + + c λ k k k k j k j j, For ths polynomal product to be stable all the factor polynomals Pj ( s) need to be stable,.e. Hurwtz, for every λ, egenvalue of ( L + G). Usng the algebrac Rccat equaton j A P + PA + Q PB R B P =, C C C C 37

151 for the k-th order system wth matrces AC and BC A C, = B C = the optmal feedback (85) wll stablze the system (84). ext we use the complex gan margn regon assessed usng the Lyapunov equaton, where P s the soluton of the Rccat equaton ( A + zb K) P + P( A + zb K) = A P + PA + z K B P + zpb K * C C C C C C C C = A P + PA Re zpb R B P C C C C = Q + < ( Re z) PBC R BC P, to guarantee an unbounded gan margn regon n complex plane, 3. herefore, for Re z suffcently large one has stablty, meanng that the characterstc polynomal of A C zb K C s Hurwtz. But the characterstc polynomal of A zb K equals (87), wth z nstead of j. C C λ hs concludes the proof. Remark 6.4: he observer coeffcents c ck can be read off from the optmal sngle nput feedback. hs allows for a farly smple desgn procedure for the system (8). Also, ths means that the complex gan margn regon for the Rccat feedback s contaned n a regon of complex plane defned as n n { C ( )... stable} S = z P s = s + c zs + + c z c z n. generalzng the concept of synchronzng regon to systems descrbed by polynomals. 6.5 Synchronzaton to the Leader Drven by an Unknown Input hs secton gves a specfc applcaton of the results of prevous sectons on a multagent system wth lnear tme-nvarant agents havng a leader drven by an nput. Addng an nput to the leader enlarges the set of possble trackng commands x ( t) one acheves synchronzaton to. Also, because of the way ths nput effects the system t satsfes the 38

152 matchng condton, 46, and can be deally completely compensated usng the approprate control System Dynamcs Assume the leader node dynamcs to be of the form x = Ax + Bu, (88) and the set of dentcal agents havng dynamcs x = Ax + Bu, (89) wth, n ; p x x,. he global synchronzaton error dynamcs reads R u, ; u R whch n global form equals δ = Ax Ax + Bu Bu = Aδ + B( u u ), (9) = ( I A) + I B( u u ). (9) δ δ Pck the dstrbuted cooperatve lnear feedback control (6) u = cke, (64). Such choce of dstrbuted control guarantees synchronzaton wthout the leader's nput, u =, for suffcently large, as detaled n Lemma 6.5 and 3. In global form ths leads to u = c( L + G) K( x x ) = c( L + G) Kδ, (9) and the global synchronzaton error dynamcs equals δ = ( I A) δ c( L + G) BKδ I Bu δ = A δ I Bu c c >. (93) Even though A c can be assumed stable under condtons of Lemma 6.5, the system (93) s not autonomous so one cannot guarantee that lm =,.e. that the systems (89) reach t δ synchronzaton asymptotcally. In fact the second term n (93) can be nterpreted as dsturbance ξ = Bu, arsng solely from the leader's nput. hat dsturbance acts only on the 39

153 leader, but ts effect s dstrbuted to all the agents. If u s unformly bounded, asymptotc A c stablty of guarantees unformly ultmately bounded convergence of the system (89), Estmatng the Leader's Input One soluton to the leader's nput problem s addng an addtonal compensatng nput, equal to u, to each agent, n order to compensate for the dsturbance. hs leads to the control law u = cke + u, (94) so that the -th agent s dynamcs equals x = Ax + Bu + cbke. (95) Such a choce makes the drft part of the agents dynamcs Ax + Bu equal to that of the leader node. hs results n a δ -system δ = ( I A) δ c( L + G) BKδ + I Bu I Bu δ = A δ c, (96) whch s an autonomous system that, under the assumpton that A c be asymptotcally stable, leads to synchronzaton; lmδ. he problem wth such a choce s that t requres to t = u u be known to all agents. hat means, ether pnnng to each agent or off-lne stored at each agent. ether of these choces seems appealng. However, a modfed control law nvolvng the estmator of the type ntroduced n Secton 6.4, wth a correcton for every agent n the form u u = cke + w (97) leads n global form to 4

154 δ = ( I A) δ c( L + G) BKδ + I Bw I Bu δ = Acδ + I B( w u) δ = A δ + I Bδ c w (98) [ ] p where w = w, and s the estmaton error. he augmented w R δ w = w u system can be wrtten as δ Ac I B δ d δ w δ w = A, (99) w dt n n δ w δ w where t s mplctly assumed that δ w satsfes a, possbly hgh order, autonomous lnear A dynamcs descrbed by the system matrx. Gven the stablty of, one can guarantee w lmδ = f also lmδ =. But the latter s the consensus problem for the estmates t t w w u. Crucal problem s assurng the convergence of all observaton to the A c w u common value equal to. In order to solve ths observaton consensus problem one uses the control communcaton graph, or more generally some subgraph of t, stll contanng a drected spannng tree snce ths s necessary for convergence. Here we practcally use the results of Secton 4 to estmate the leader's dsturbance sgnal. u = ( k ) Under the condton dentcally the hgher order observer (8) k j ( k ) d w ( ) (, () = c j el wl w g u w ) j j dt + = l or n global form k ( k ) ( j) = j ( + ) p w j= w c L G I δ, () 4

155 gves asymptotc estmate convergence for the proper choce of coeffcents c j e.g. as detaled n heorem 6.3. One should note that the ncreasng order k requres observers of ncreasng complexty. he estmaton error dynamcs δ w () satsfes the assumpton on the lnear form of the estmaton error system (99). If, however, one has the leader's nput u modeled as an output of an autonomous lnear system,.e. the command generator, gven n state-space form as F wth the system matrx, and the output matrx, of an approprate order, one can use the observaton-synchronzaton algorthm (79) n the followng form v. Fv cl = + ej ( wj w ) + g ( u w ) j v u w = Fv = Hv Synchronzaton of, defned for every agent, mples the output synchronzaton of,, H = Hv v w meanng δ = w u = I H( v v ), whch n turn guarantees the state synchronzaton; w lmδ. One has the synchronzaton of estmates f the matrx I F c ( L + G) L H s t = v Hurwtz. In ths case also the assumpton on the lnear form of the estmaton error δ v = v v dynamcs (99) s satsfed. Agan t should be remarked that for more complcated form of the control nput, u requrng ncreasng dmenson of the state-space model for u, the correcton algorthm becomes ncreasngly complcated. 4

156 6.6 Applcaton to Second-order Double-ntegrator Systems Wth Dsturbances hs secton presents another applcaton of dsturbance estmaton. Here we are concerned wth the second-order double-ntegrator systems wth dsturbances. hs model of prmarly moton systems s often studed n the lterature, 48,49,49. Let the mult-agent system consst of a leader x = v v = u + ξ, () and agents of the form he dsturbances n ths case satsfy the matchng condton. x = v v = u + ξ. (3) Let the agents' dsturbances ξ be modeled as (7), or more precsely as an output of (7) that s not drectly measurable, e.g. as detaled n the numercal example, Secton 6.9. ote that ths s more general than the case presented n Subsectons 6.4. and 6.4. where t s assumed that the entre dsturbance state acts on the agent dynamcs. Measurable output of the dsturbance generator (7), ς, s used for dsturbance system state estmaton. Let the leader's dsturbance be modeled as (78) or have the dentcally vanshng dervatve of a fnte ξ = ( k) order,. Here one apples the results of Secton 6.4, usng the dstrbuted observer (79) or (8) for the leader's dsturbance and local observers (7) for agents' dsturbances. he dstrbuted observer (73) s applcable f some agents, root nodes n a spannng tree, know the fxed zero reference, 3. he agents that have the leader state pnned nto them are consdered here to know the fxed zero reference, 3. he estmates of and ξ are computed from (7) or ξ (73) and (79) or (8) respectvely. hose are then used as the compensatng sgnal producng the total control sgnal 43

157 he control gans are determned by the local Rccat desgn, as detaled n Lemma 6.5, [ ] K= c c, and the observer gans are desgned as detaled n Lemma 6.6, and heorem 6.3. So by results of Lemma 6.5, 6.6, and heorem 6.3, one has estmate convergence for the agents' and the leader's dsturbances together wth the asymptotc state synchronzaton under control (4). 6.7 Applcaton of Dsturbance Compensaton to Dynamcal Dfference Compensaton hs secton nvestgates the effects of unmodelled dynamcs and possbly heterogeneous agents' dynamcs both descrbed as state dependent dsturbances actng on the nomnal system. Gven the agent's dynamcs n control affne form wth all agents havng the same order as dynamcal systems, one can assume the exstence of a nomnal drft dynamcs f ( x ) havng the same order as f ( x ),. Each agent can then be descrbed as In the case a leader s present, havng dynamcs gven by, x ( ) ( ) ( ) ( ). (5) = f x + g x u + f x f x the choce of the nomnal dynamcs s suggested by (6), he error system n that case equals { ( ) ( ) ( ) ( ) } u = e c x x + g x x + c v v + g v v ˆ ξ + ˆ ξ. (4) j j j j c, c x = f ( x ) + g ( x ) u [ ] no mn al sstem ξ ( x ) x = f ( x ), (6) [ ] x. (7) = f( x ) + g ( x ) u + f ( x ) f( x ) no mn al sstem ξ ( x ) δ = f ( x ) f ( x ) + g ( x ) u + ξ ( x ). (8) 44

158 Assumng Lpshtz contnuty of f, or assumng dfferentablty and applyng contracton analyss, allows one to assess the contrbuton of the nomnal nonlnear drft part, f ( x ) f ( x ), δ n terms of. ote that the dynamcal dfference of the leader and an agent under consderaton s decomposed nto two parts f ( x ) f ( x ) = f ( x ) f ( x ) + ( f ( x ) f ( x )), frst part begn due to the state dscrepancy from consensus,.e. δ and the second one due to the nherent dfference n dynamcs. If one can fnd dstrbuted feedback control laws,, for all agents such that the nomnal agents reach consensus asymptotcally then the earler conclusons can be appled to the dsturbance term u ( x) ξ ( x ) = f ( x ) f ( x ) (9) Furthermore, the compensatng nput can be used, snce the nomnal system s agan affne n control u = u + ω, such that the sze of the effectve dsturbance g ( x ) ω + ξ s mnmzed. It s clear that f the dynamcal dfference ξ ( x ) = f ( x ) f ( x ) satsfes the matchng condton, 46, then the compensatng term ω can cancel t completely. In real stuatons dynamc dfference observers should be used, 47. An example of such a case s a set of feedback lnearzable agents all havng the same (full) relatve order. As long as the same agent-state-space transformaton, preservng partal stablty notons as detaled n Lemma 6.4, brngs all agents to the nonlnear controllable canoncal form, the dynamcal dfferences can be cancelled by usng transformed control sgnals. Such an acton preserves the consensus manfold. Yet another nterestng example s furnshed by the specal case of agents havng the followng form S 45

159 x = Ax + Bu + ξ ( x ). hus, all agents have dentcal lnear dynamcs ( A, B), but the nonlnear unmodelled dynamcs may be dfferent. One can use lnear dstrbuted synchronzaton control (6) to guarantee the synchronzaton of the nomnal system. If the dsturbances are unformly bounded ξ ( x ) < M, the unformly ultmately bounded consensus result readly follows. If, on the other hand, the unmodelled dynamcs s the same,.e. ξ ( x ) = ξ ( x ), (, j), then t satsfes x S ξ ( x) S, x j where S s the consensus affne manfold. hus the consensus manfold remans an nvarant manfold for the mult-agent system. If the contrbuton of the unmodelled dynamcs satsfes unform bounds of heorem 6. robustness of the lnear part of the system guarantees synchronzaton for the nonlnear system. Such s the partcular case of unformly contnuous functons, though ths condton s not necessary. State dependent unformly contnuous S dsturbances ξ ( x ), lumped together nto a global vector, preserve the nvarance of the consensus manfold S and satsfy the growth bound unformly along the manfold. A specfc example of ths nstance s the topc of the next secton. 6.8 Example of State Dependent Dsturbances-Agents' Dynamcs Satsfyng the Unform Growth Bound hs secton presents an example of a mult-agent system wth dentcal agents havng Lenard dynamcs. Such systems are guaranteed to have lmt cycles. he Lenard dynamcal system descrbes an oscllator, havng -dmensonal state space R, wth an nput. he correspondng dfferental equaton has the form y + αφ( y) y + ψ ( y) = u () 46

160 R In the state space wth state varables x = y, x = y + αφ( y), where ( x ) ( ) d, the equaton () can be wrtten as a system, d x x αφ( x ). () dt x = ψ ( x ) + u he lmt cycle exsts f ψ ( y) > for y >, and lm Φ ( y) =. If ψ ( y) = y one can separate the lnear and nonlnear parts of the system as d x x ( x ) α Φ u. () dt x = + x y x Φ = φ ξ ξ If the nonlnear functon Φ( x ) s unformly contnuous, whch s accomplshed for unformly φ y y bounded ( y), negatve for small, and postve for large, conclusons of prevous paragraphs become applcable. akng for example φ( y) = y y + one has Φ ( x ) = x ξ dξ. A lnear synchronzaton algorthm desgned for lnear part of the ξ + system, 3, guarantees exponental partal stablty wth respect to the consensus manfold, wth the pertanng robustness property. As the nonlnear parts are the same for all agents ther contrbuton to the partal stablty Lyapunov functon vanshes on the consensus manfold, and snce the lumped vector of the nonlnear terms comprses a unformly contnuous functon, all components beng unformly contnuous, one can state that there exsts a growth bound vald unformly along the consensus manfold. he sngle agent system can be concsely wrtten as x = Ax + ( x ) + Bu ( x). (3) αξ 47

161 Mult-agent system has agents of the form (3), wth u ( x) the feedback cooperatve control for agent. One can choose the lnear local neghborhood error cooperatve control (6), wth the feedback gan K desgned usng local Rccat desgn for the lnear system ( A, B). Snce Φ( x ) s unformly contnuous wth a bounded dervatve one has the followng bound Φ( x ) Φ( x ) M x x = = Φ Φ Φ = = = = d ( ( x), S) ( x ) ( x ) M x x M d ( x, x ) M d ( x, S), (4) f x s chosen so that t mnmzes the dstance of the state from the consensus manfold,.e. d( x, S) = d( x, x ). M s a postve constant satsfyng M. So the dsturbance stemmng from the nonlnear part of the dynamcs satsfes the unform growth bound (45). If one had a leader of the form x = Ax +αξ ( x ) (5) he same concluson apples, but now the system n queston s the error system δ = x x δ = Aδ + Bu( δ ) + α( ξ( x ) ξ( x )) (6) Usng agan (6) wth the local neghborhood error e modfed approprately for the pnnng terms one obtans the closed loop system n global form δ = A δ + α( Ξ( x) Ξ( x )), (7) cl where Ξ ( x) = ( ) ( ) ξ x ξ x. Snce one can assume the nomnal system to be exponentally stable there s a classcal quadratc Lyapunov functon V = δ Pδ and wth the growth bound Ξ( x) Ξ( x ) M δ one has exponental stablty gven that the convergence 48

162 rate of the nomnal system s suffcently large. he proof of the growth bound (46) n ths case s dentcal to (4) wth x beng the leader's state. Φ( x ) Φ( x ) M x x = = = = = Φ( x) Φ( x ) Φ( x ) Φ( x ) M x x = M δ In the case of cooperatve regulator problem desgnng the lnear dstrbuted synchronzaton algorthm u = cl Kx for the system x = Ax + Bu and fndng the quadratc partal stablty Lyapunov functon V ( x) = x Px for t, wth dstance convergence suffcently large to overpower the dsturbance contrbuton (ths begn measured by the matrx Q ) (67), guarantees synchronzaton of the orgnal nonlnear system globally. ote that these results are dfferent than conclusons based smply on the lnearzaton, snce the lnearzaton guarantees dstance convergence,.e. synchronzaton, only locally, whle the Lyapunov method gves an estmate of the attracton regon to an apprecable dstance from the consensus manfold S. For all ntal states n ths regon of attracton, the trajectory converges to the consensus manfold, and n partcular to the lmt cycle there. For the feedback control sgnals u( x) vanshng on the consensus manfold the dynamcs on the consensus manfold equals the dynamcs of a sngle free,.e. uncontrolled, agent. Snce all the ndvdual free agents' lmt cycles, homeomorphc to S, comprse the - torus = S S S n the product total space ths s an nvarant set for the nonnteractng agents. he resultng lmt set for nteractng agents wll be the ntersecton of the R wth the consensus manfold S. hs s a subset of the nvarant set for non-nteractng agents. he fact that a dsspatve dstrbuted consensus algorthm, u( x), changes the topologcal nature of the nvarant sets for the closed loop system, as compared to the nvarant 49

163 sets for the aggregaton of free (not controlled) systems, can be consdered an onset of complex behavour. hs qualtatvely dfferent, complex, behavour was absent when the agents were not nteractng. 6.9 umercal Example hs secton gves numercal examples of the dsturbance observaton schemes and the control protocols ntroduced n ths chapter, n partcular Secton 6.6. Consder the leader and 5 agents havng the double ntegrator form (), (3), d x, x, u., v = dt, v + +, ξ, he leader's dsturbance s modelled as ξ ( t ) = 3sn.63 t sn.63 t (5) 9, so ξ. he agents' dsturbances ξ are modelled as an output of the dsturbance generator that s not drectly measurable. It s assumed that the output ς s drectly measurable and t s used n the dsturbance observer protocols. y =.5 y ξ = y ς = [ ] [ ] All agents' dsturbances are modelled by the same model, thought wth dfferent, random, ntal condtons, so both local (7) and dstrbuted (73) observer shall be appled. he nterconnecton graph and pnnng gans are gven by 3 3 L = 3, G = dag(,,,,). Frst we observe the behavor of the mult-agent system when dsturbances are not compensated, that s under the dstrbuted synchronzaton control for the nomnal systems. 5 y

164 hen we compensate for the leader's dsturbances assumng t alone acts on the system. Further, the agents' dsturbances are allowed to act as well. hey are estmated usng both local and dstrbuted observer, comparng the synchronzaton performance n both cases. For the desgn of the synchronzaton control law (4) we chose the local feedback gan usng the [ ] K = c =.5 Rccat desgn (63).36.75, wth the couplng constant. hs corresponds to the choce Q =.5 I, R = n (64). For the observers we made the followng choces. he local observer gan s L = [ ] whch s obtaned from the observer Rccat equaton (75), (76). hs corresponds to the choce Q = I, R = n (76). he same observer gan was used for the dstrbuted agents' dsturbance observer (73) wth the couplng constant c =. he dstrbuted leader nput observer coeffcents were chosen accordng to the Rccat desgn detaled n heorem 6.3, wth Q =. I, R =. hs gves the necessary 5 coeffcents as. Coeffcents were delberately chosen to gve slow convergence so that the effects of leader's nput estmaton would be dscernble. Fgure 6. depcts the generated agents' dsturbance sgnals, whle Fgure 6. and 6.3 show the observer errors for local and dstrbuted observers. he observers are seen to converge to the true values of agents' dsturbances snce ther observer errors converge to zero. Small perodc devatons of observer errors from zero seen n Fgure 6. and 6.3 are artfacts of computng errors. [ ] 5

165 5 Agent's dsturbances Fgure 6. Dsturbances actng on agents ξ... ξ 5 3 Local agent dsturbance errors Fgure 6. Agents' dsturbance local estmate errors 35 Dstrbuted agent dsturbance observer Fgure 6.3 Agents' dsturbance dstrbuted estmate errors Fgure 6.4 gves a comparson of observer convergence for local and dstrbuted observers of agents' dsturbances. In ths partcular nstance the local observers converge faster then the dstrbuted observer. hough generally ths need not be the case, and the use of a local 5

166 versus a dstrbuted observer s dctated by whether t s possble to measure absolute or only relatve quanttes Comparson of local and dstrbuted agent dsturbance observer errors Fgure 6.4 Comparson of local and dstrbuted agents' dsturbance estmate errors he leader's dsturbance sgnal, whch s also nterpreted as an nput, s gven together wth the agents' estmates of the same n Fgure 6.5. All agents' estmates are seen to converge to the leader's nput sgnal. 5 Dstrbuted leader nput estmates Fgure 6.5 Leader's nput dstrbuted estmates 53

167 he remanng fgures show the dynamcs of the mult-agent system,.e. the frst states of all agents, n dfferent crcumstances. Fgure 6.6 shows the case where the leader's nput s not compensated, and the agents' dsturbances do not act on the system. Synchronzaton s not acheved snce the leader's dsturbance prevents t. otce that the effect of leader's dsturbance gets dstrbuted to all the followng agents. hen, the leader's dsturbance s estmated and compensated whch results n synchronzaton to the leader's trajectory as shown n Fgure 6.7. Faled synchronzaton, no leader nput compensaton, no agent dsturbances Fgure 6.6 Agents' frst states, agents dsturbances do not act, there s no leader nput compensaton- synchronzaton to the leader s not acheved Synchronzaton-leader nput compensated Fgure 6.7 Agents' frst states, leader's nput s estmated and compensated, agent dsturbances do not act, synchronzaton to the leader s acheved However, when the dsturbances act on the agents the synchronzaton s not acheved, as depcted n Fgure 6.8. Fgures 6.9 and 6. show cases of synchronzaton when the agents' 54

168 dsturbances are also estmated and compensated. Frst by the local observers, depcted n Fgure 6.9, then by the dstrbuted observers, depcted n Fgure Leader nput compensated-agents dsturbances present Fgure 6.8 Agents' frst states, leader's nput s compensated, but dsturbances actng on the agents are not compensated. he synchronzaton s not acheved 3 Synchronzaton-agents dsturbances locally estmated Fgure 6.9 Agents' frst states, leader's nput s compensated, agents' dsturbances are locally estmated and compensated. Synchronzaton s acheved. 55

169 4 Synchronzaton-agents dsturbances dstrbutvely estmated Fgure 6. Agents' frst states, leader's nput s compensated, agents' dsturbances are dstrbutvely estmated and compensated. Synchronzaton s acheved. 6. Concluson o conclude, ths chapter presents varous methods of dsturbance estmaton n multagent systems for the dsturbances actng on the leader and on the followng agents. If the nomnal system reaches synchronzaton exponentally then t possesses a type of cooperatve robustness whch can be quantfed by a Lyapunov functon. However, n many cases such robustness alone does not guaranty that the control goal of synchronzaton s attaned when dsturbances are present. Dsturbance estmators are used to guarantee the control goal,.e. synchronzaton, even n presence of dsturbances. Specal applcatons were consdered; the case of there beng an nput drvng the leader, whch needs to be dstrbutvely observed by all the followng agents, the case of second-order double-ntegrator systems wth dsturbances actng on the leader and the agents, and the case of heterogeneous agents. Computer smulatons show effectveness of the proposed observaton schemes and control desgn methods on the example of second-order double-ntegrator systems. hs offers a comparson of mult-agent system performance wth and wthout dsturbance compensaton. It can be seen from the presented example that good qualty dsturbance estmaton and compensaton s a necessary prerequste for hgh performance mult-agent systems. 56

170 CHAPER 7 COOPERAIVE OUPU FEEDBACK FOR SAE SYCHROIZAO 7. Introducton he last two decades have wtnessed an ncreasng nterest n mult-agent network cooperatve systems, nspred by natural occurrence of flockng and formaton formng. hese systems are appled to formatons of spacecrafts, unmanned aeral vehcles, moble robots, dstrbuted sensor networks etc.,,3,4,5,6,7,8,9. Early work wth networked cooperatve systems n contnuous and dscrete tme s presented n,,3,4,6,7. hese papers generally referred to consensus wthout a leader. We call ths the cooperatve regulator problem. here, the fnal state of consensus depends on ntal condtons. By addng a leader that pns to a group of other agents one can obtan synchronzaton to a command trajectory usng a vrtual leader, also named pnnng control, 8,3,9. We call ths the cooperatve tracker problem. In the cooperatve tracker problem all the agents synchronze to the leader's reference trajectory. ecessary and suffcent condtons for synchronzaton are gven by the master stablty functon, and the related concept of the synchronzng regon, 8,,. For contnuous-tme systems synchronzaton was guaranteed,,3, usng local optmal state-feedback derved from the algebrac Rccat equaton. It was shown that, usng Rccat desgn of the local feedback gan for each node guarantees an unbounded rght-half plane synchronzaton regon n the s-plane. hs allows for synchronzaton under mld condtons on the drected communcaton topology. However, the entre state s not always avalable for the feedback control purposes. In such nstances one possble soluton s to use a dynamc output-feedback, as detaled n,3,8,3. here, the observer based dynamc cooperatve regulator was used to guarantee state synchronzaton. hs chapter nvestgates condtons under whch state synchronzaton s asymptotcally acheved by statc output-feedback. Statc output-feedback s smpler, and more easly mplementable than dynamc output-feedback. o addtonal state observaton s needed. 57

171 he structure of the chapter s as follows, Secton 7. presents the mult-agent system dynamcs and defnes the control problem. Secton 7.3 derves suffcent condtons on the dstrbuted output control that guarantee the control goal. Followng sectons expand on the results of Secton 7.3. amely, Secton 7.4 descrbes the dstrbuted output-feedback n twoplayer zero-sum game context. Suffcent condtons are derved that allow for a soluton of the global two-player zero-sum game by the dstrbuted output-feedback. Secton 7.5 specalzes results of Secton 7.4 to the case where the dsturbance sgnal s absent,.e. the cooperatve globally optmal output-feedback control problem. Condtons are shown, under whch the consdered dstrbuted output-feedback control s optmal wth respect to a quadratc performance crteron, mplyng state synchronzaton. Secton 7.5 can be seen as complementng the work presented n Chapter 4 that dscusses global optmalty of cooperatve control laws for consensus and synchronzaton n the full-state feedback case. Secton 7.6 examnes the condton on the graph topology appearng n Sectons 7.4 and 7.5, and dentfes the key property of the graph matrx that satsfes that condton. Conclusons are presented n Secton Synchronzaton Wth Dstrbuted Output Feedback Control he mult-agent system under consderaton s comprsed of dentcal agents and a leader, havng lnear tme-nvarant dynamcs. Let the leader system be gven as x y = Ax, = Cx, (8) and the followng agents as x = Ax + Bu, y = Cx. (9) 58

172 Defnton 7.: he dstrbuted synchronzaton control problem, for a mult-agent system (9) wth a leader (8), s to fnd dstrbuted feedback controls, u, for agents, that guarantee lm x ( t) x ( t) =,. We call ths the cooperatve tracker problem. t Defne the local neghborhood error he feedback control sgnal for agent s chosen as e = e ( x x ) + g ( x x ), () j j j u = ckce. () he expresson e = Ce = e ( y y ) + g ( y y ), () y j j j s the local neghborhood output error. Let δ = x x be the synchronzaton error. In global form one has = δ δ δ, so e = ( L + G) I, (3) nδ whch appeared n 6,8. Gven that the graph contans a spannng tree wth at least one non zero pnnng gan connectng nto a root node, the matrx L + G s nonsngular, 6, therefore e = δ =. In global form, the local neghborhood output error, (), equals e = ( L + G) Cδ. (4) y Expresson (4) reflects the constrant on the nformaton avalable for dstrbuted feedback control. he communcaton topology, determned by the matrx L + G, and the structure of the output matrx C determne whch states, and whch parts thereof, can be used for dstrbuted control purposes. he choce of feedback () gves the closed loop agent system as x = Ax + cbkce 59, (5)

173 whch yelds the dynamcs n global form he synchronzaton error global dynamcs follows as x = ( I A) x c( L + G) BKCδ. (6) δ = ( I A c( L + G) BKC) δ. (7) Lemma 7.. he matrx I A c( L + G) BKC s Hurwtz f and only f all the matrces A cλ BKC j are stable, where λ are the egenvalues of the graph matrx ( L + G). j Proof: Upon applyng the state transformaton, I n, where ( L + G) = Λ s a trangular matrx, to I A c( L + G) BKC, one obtans the system matrx I A cλ BKC. (8) he block-dagonal elements of (8), A cλ BKC, determne the stablty of the orgnal j matrx, I A c( L + G) BKC. he result of Lemma 7.. allows for the nterpretaton of the stablty propertes of system (7) n the context of robust stablzaton for a sngle agent system. Before proceedng further, a useful concept of a synchronzng regon for output feedback s ntroduced. he noton of the synchronzng regon for output feedback appeared n 8. Defnton 7.: Gven matrces ( A, B, C), and the feedback gan K, the synchronzng regon for the matrx pencl A σ BKC s a subset of the complex plane, y { σ C : σ s stable} S = A BKC cλ j. hs we call the synchronzng regon for output feedback. Hence, the matrx (8) s stable f and only f all the scaled graph matrx egenvalues,, are n the synchronzng regon for output feedback of the matrx pencl A σ BKC. (9) 6

174 Wth the system ( A, B, C) consdered as gven, the synchronzng regon for output feedback, S y, depends on the choce of the local output feedback gan K. 7.3 Local Output Feedback Desgn hs secton nvestgates a specfc choce of the local output feedback gan, K, and the pertanng synchronzng regon for output feedback. he chosen local output feedback gan s determned by the soluton of the output algebrac Rccat-type equaton, 66, as detaled n the followng subsecton Output Feedback Gan Let the local output feedback gan K satsfy the followng relaton for some matrx M, KC = R ( B P + M ), (3) where the matrx P = P > solves the output algebrac Rccat-type equaton A P + PA + Q PBR B P + M R M =. (3) Frst we show that the output-feedback (3) s stablzng for the system ( A, B). hs result, presented here as a proposton, was mentoned n 66. Proposton 7.. Let the lnear tme-nvarant system be gven by matrces ( A, B, C). Let there exsts a postve defnte soluton, P = P, of the output algebrac Rccat-type equaton hen, the output-feedback, A P + PA + Q PBR B P + M R M =. (3) u = Ky = KCx, (33) wth the output gan, K, satsfyng KC = R ( B P + M ), (34) for some matrx M, s a stablzng output feedback for the matrx A BKC. 6

175 Proof: ake the quadratc Lyapunov functon, V ( x) = x Px, for the system determned by the matrx A BKC. he matrx P > s chosen as a soluton of the output algebrac Rccattype equaton (3). he tme dervatve of ths Lyapunov functon s determned by the matrx ( A BKC) P + P( A BKC) = A P + PA C K B P PBKC = A P + PA + KC R B P R KC R B P C K RKC PBR B P ( ) ( ). Wth the choce of the output feedback gan satsfyng (34), ths becomes ( ) ( ) A BKC P + P A BKC = A P + PA + M R M C K RKC PBR B P = Q C K RKC <, hence the stablty of the matrx A BKC s guaranteed he Guaranteed Synchronzng Regon for Output Feedback For the complex matrx pencl, A σ BKC, one adopts a smlar approach to fnd the guaranteed synchronzng regon for output feedback. hs motvates the followng theorem. heorem 7.. Let the mult-agent system be gven by (8), (9). Let the graph have a spannng tree wth at least one non-zero pnnng gan connectng to a root note. Choose the dstrbuted statc output feedback control (), where the output feedback gan, K, satsfes (34). Let the followng abbrevatons be ntroduced a = Q C K RKCQ >, (35) / / : λ ( ) mn λ mn KCQ / where denotes the smallest egenvalue on the complement of the kernel of, / / / / b : Q C K MQ Q M KCQ = +, (36) and A : = 4( a b ), A : = 4( a a + b ), A : = 4 b, A : = ( a) ( b). 3 4 (37) he form of the guaranteed synchronzng regon for output feedback depends on the matrx M, n that 6

176 . he guaranteed synchronzng regon for output feedback s an nteror of an ellptc A A4 > 4A regon n C f A and. <. he guaranteed synchronzng regon for output feedback reduces to the empty set f A < A A4 < 4A and.. he guaranteed synchronzng regon for output feedback s a hyperbolc regon n C, havng two connected components, f A > and. v. he guaranteed synchronzng regon for output feedback s a hyperbolc regon n C, A havng a sngle connected component f and A. > A4 > 4A Proof: ake the quadratc Lyapunov functon, V ( x) A A4 < 4A = x Px, for the complex system (9). he postve defnte real matrx, P = P, s chosen as a soluton of the output algebrac Rccat-type equaton (3). he tme dervatve of ths Lyapunov functon s determned by the matrx ( A σ BKC) P + P( A σ BKC) = A P + PA σ C K B P σ PBKC, whch can be further wrtten, by completng the squares, as A P + PA σ C K B P σ PBKC ( σ ( )) ( σ ( )) = A P + PA + KC R B P + M R KC R B P + M σ C K RKC + σ C K M + σ M KC C K RKC. he choce of the output feedback satsfyng (3) makes ths expresson equal to A P + PA + σ C K RKC σ C K RKC + σc K M + σm KC C K RKC. Usng the output Rccat-type equaton (3), ths can be wrtten as 63

177 + Q PBR B P M R M C K RKC + ( Re σ ) C K RKC + σ C K M + σ M KC = ( + ) ( + ) + Q B P M R B P M M R M PBR B P + ( Re σ ) C K RKC + σ C K M + σ M KC. he suffcent condton guaranteeng the synchronzng regon for output-feedback then becomes Q ( B P M ) R ( B P M ) M R M PBR B P + ( Re σ ) C K RKC + σ C K M + σ M KC <, allowng the assessment of the output synchronzng regon n C. In partcular, ths expresson s equvalent to Q σ C K RKC + σ C K RKC + ( σ ) C K M + ( σ ) M KC < whence one fnds the suffcent condton for the synchronzng regon for output-feedback to be Q (Reσ ) C K RKC + ( σ ) C K M + ( σ ) M KC < ( I + (Re σ ) Q C K RKCQ ) > ( σ ) Q C K MQ + ( σ ) Q M KCQ hs nequalty s certanly satsfed f / / / / / /.(38) λ σ / / ( I + (Re ) Q C K RKCQ ) mn > ( σ ) Q C K MQ + ( σ ) Q M KCQ * / / / /, where λ mn denotes the smallest egenvalue on the subspace complement of the kernel of / KCQ /, snce on the ker( KCQ ) the nequalty (38) reduces to the tautology, I >. he propertes of the matrx norm for Hermtan matrces gve an nequalty, ( σ ) Q C K MQ + ( σ ) Q M KCQ * / / / / σ + / / / / ( Q C K MQ Q M KCQ ), whch lends tself to gve a conservatve condton guaranteeng the synchronzng regon for output-feedback, 64

178 / / λ ( I + (Reσ ) Q C K RKCQ ) mn (39) / / / / / / = + (Reσ ) λ ( Q C K RKCQ ) σ ( Q C K MQ + Q M KCQ ). mn he conservatve nature of the stablty condton (39), n ths case, s the prce to pay for smplcty. Introducng the abbrevatons (35),(36), renders the condton (39) for the synchronzng regon for output-feedback to a consderably smpler form + ( Reσ ) a > σ b. (4) he geometry n C determned by ths expresson can be elucdated by explctly calculatng the bound for σ = x + jy C. Assumng + ( Reσ ) a >, whch, gven postveness of b, s necessary for the nequalty (4) to hold, one can square both sdes to obtan an equvalent nequalty hs equals ( + ( Reσ ) a) > 4 σ + x a > x + y b ( ( ) ) 4(( ) ), + a x + x a b x b y > ( ) ( ) 4 ( ) 4, and after some straghtforward algebrac manpulatons one obtans an expresson 4( a b ) x + 4( a a + b ) x 4 b y + ( a) ( b) >. At ths pont, for the sake of smplcty, further abbrevatons, (37), are ntroduced yeldng the A x + A x A y + A > x 3 4 expresson,. Fnally, by completng the squares n, one fnds the geometrcal meanng of the nequalty (4), expounded as A A x A y A ( + ) 3 > 4 + A 4A b A.. (4) Dependng on the sgns of A, A 4 A 4A ths s ether an unbounded hyperbolc regon n the complex plane or the nteror of an ellptc regon. Snce A, f one has an 3 A = 4( a b ) < 65

179 A A nteror of the ellptc regon for A >, or an empty set for A <. If, alternatvely, 4 4 4A 4A A a b = 4( ) > one has a hyperbolc regon. hs regon has two connected components f A A A <, or a sngle connected component f A > A 4A In case ) of heorem 7., the guaranteed synchronzng regon for output-feedback reduces to an empty set. However, the stablty condton (4) s conservatve, hence ths does not mean that the actual synchronzng regon s empty. By the result of Proposton 7., gven the output-feedback gan (34), the complex number σ = s an element of a synchronzng regon for output-feedback, therefore the latter cannot empty. Fgure 7. he connected and dsconnected hyperbolc regons of heorem 7.. Remark 7.: otce that n case M = one recovers the unbounded synchronzng regon characterstc of the full-state feedback,,3, ( Re ) Q + σ C K RKC <. A key observaton that one should make s, M = b =, and by contnuty, b grows wth M. herefore, for M small enough, n terms of the matrx norm, one has a hyperbolc unbounded synchronzng regon for output-feedback. As M grows ths hyperbola can turn nto an ellpse, 66