Minimal Interconnection Topology in Distributed Control Design

Transcription

1 Minimal Interconnection Topology in Distributed Control Design Cédric Langbort Vijay Gupta Abstract We propose and partially analyze a new model for determining the influence of a controller s interconnection graph on closed-loop performance in distributed control design problems Given a plant composed of dynamically uncoupled subsystems, we investigate the existence and properties of an optimal structured controller, when one of the weight matrices of a classical LQR cost function is topology-dependent We also give some theoretical results as to the existence of critical prices at which adding supplementary edges becomes detrimental to closed-loop performance I INTRODUCTION The question of optimal decentralized or structured control design for systems composed of interconnected subsystems has been widely studied at least since the seventies (see eg, [15], [3]) The defining feature of these problems is that, while the coupling due to the cost function to be optimized can demand that the control of one subsystem know the states of all the others, the topology (or information pattern) imposed by the interconnection may not allow such interactions to happen A lot is known about the information patterns for which an optimal structured control law exists [19] or has some desirable properties, such as being linear and satisfying a separation principle [20], [15] or being computable in polynomial time in the dimension of the plant s data [4], [14] Many synthesis methods are also available that can impose a specific topology on a controller (see, eg, [10] and the references therein), result in control laws that can be proved to be decentralized a posteriori [1], [5] or give an approximation to the exact optimal structured controller (see [6] and the references therein) In all the works mentioned above the controller s interconnection topology is always (if sometimes implicitly) assumed to be known to the designer prior to synthesis, and optimization and/or design are to be performed among control laws with this particular structure While this problem formulation is appropriate when decentralization is viewed as an external constraint (eg, when controlling a system with a pre-existing interconnection topology such as a power distribution network), there are situations where the communication network and the controller are both designed at the same time For example, when deciding between a leaderfollower and a fully decentralized architecture for cooperating multi-vehicle systems, the choice of information pattern is an integral part of the control design process In such cases, C Langbort is with the Center for the Mathematics of Information, California Institute of Technology 1200 E California Ave MS 136/93, Pasadena CA clangbort@istcaltechedu V Gupta is with the Department of Electrical Engineering, California Institute of Technology vijay@cdscaltechedu it makes sense to try to find the minimal (with respect to some appropriate cost function) topology needed to achieve a particular control goal This structural optimization problem is complementary to other, communication theoretic, tradeoffs arising in network/ distributed controller co-design (as discussed eg, in [11]) and falls within the framework of the theory of organizational efficiency and information cost introduced by Marschak and Radner in [12] It is surprising that, while the tools and ideas of Team Decision Theory developed in [12] have been successfully applied to distributed control problems with given information structures [8], the very question of efficiency of decision architectures, that Marschak and Radner were originally addressing, has received relatively little attention in the control literature In particular, we are only aware of the following recent attempt at finding a minimal control interconnection structure In [17], the authors have shown that, when constructing a distributed controller from a set of observer-based controllers using different and parallel observations, the star interconnection topology is minimal, in the sense that the resulting control design problem has the minimal number of free parameters needed to ensure closed-loop stability In this paper, we present a new model for studying the role of controller topology in distributed control problems, which is inspired by ideas of the field of Games over Networks in Economics [9], [16] More precisely, we propose to modify the classical structured LQR cost function by making one of the weight matrices topology-dependent This amounts to explicitly accounting for the cost of communication and will thus allow us to explore the trade-off between a controller s performance and its communication network topology The paper is organized as follows After introducing our model and various notations in Section II, we prove that, under certain assumptions, the optimal control topology according to our criterion is the fully decentralized one Then, in Section III, we also investigate the existence of critical prices at which it becomes detrimental to add edges to a pre-existing controller topology A Structured Control Laws II THE MODEL Assume we are given N linear time-invariant (LTI) subsystems described by ẋ i = A i x i + B i u i (1) for all i = 1N, with the state x i (t) and input u i (t) of each subsystem being an element of R ni and R mi, respectively, for all time t 0 In the sequel, we will write

2 n := N i=1 n i and m := N i=1 m i Each pair (A i,b i ) is assumed to be controllable which, in turn (see, eg, [6]), implies that the full system, with matrices A := diag i (A i ), B := diag i (B i ) is also controllable Let G N be the set of all undirected graphs that can be constructed by taking these agents as vertices Every graph g G N specifies a communication topology that we will use to construct specific control laws for subsystems (1) Before doing so, we must introduce the edge set E(g) and adjacency matrix A(g), defined by { 1 if (i,j) E(g) (A(g)) ij =, 0 otherwise of a graph g All the graphs we will consider have self-loops ie, that (i,i) E(g) for all i = 1N, g G N We define K m,n (g) as the space of structured m n matrices with structure imposed by g A matrix K in K m,n (g) is defined block-wise, each block K ij being a m i n j matrix such that K ij = 0 whenever (A(g)) ij = 0 Hence, if a control law is such that K(g) K m,n (g), (u 1 (t)u N (t)) T = K(g)(x 1 (t)x N (t)) T t 0 (2) then u i (t) involves the values of x j (t) only for those j such that (i,j) E(g) In the sequel, a control law satisfying (2) will be said to have structure g B Cost Functions and Value of a Graph For any given positive semi-definite matrix Q S n and control law u : t (u 1 (t)u N (t)) T, we define the familiar quadratic cost J(Q,u) := + 0 u 1 (t) u N (t) x 1 (t) x N (t) T T Q u 1 (t) u N (t) x 1 (t) x N (t) dt subject to ẋ i (t) = A i x i (t) + B i u i (t) (Equations (1)) and its optimal value x i (0) = x 0 i t 0, i = 1N J (Q) := min u J(Q,u) When a graph g G N imposes a structure on the allowed control laws, we obtain an upper-bound for J (Q), namely J (Q) Jg(Q) where J g(q) := { (3) min J(Q,u) u subject to u has structure g (4) In keeping with the spirit of the previous notations, we will write J g (Q,u) instead of J(Q,u) when the control law u at hand has structure g We are now in a position to introduce the control cost (or value 1, to follow the terminology of [9] ) of a graph Let a mapping Q : G N S n be given (recall that S n denotes the set of positive semi-definite matrices of size n) Then, the value of graph g is defined as V (g) := J g(q(g)) (5) The motivation for introducing such graph-dependent weight and cost function is the following Assume that we are interested in finding a controller minimizing the cost J(Q 0,) for some positive semi-definite matrix Q 0 If there is no restriction on the structure of the control law, the optimal controller s interconnection topology will typically be a full graph In practice, however, building and maintaining each of the graph s communication edges has a cost which, if taken into account, may make this control law less attractive It is to capture this trade-off between closed-loop performance and controller topology that we introduced an information cost [12] associated to every communication graph g Of course, there are many ways in which such a cost could be defined Our choice of a graph-dependent weight matrix Q means that we are putting a price on the amount of energy used for communication, which fits naturally in the LQR framework Below are a few examples of possible maps Q, along with their physical interpretation Edge separable without interference: the map Q satisfies Q(g) := Q 0 + P ij, (6) (i,j) E(g) i j with each matrix P ij > 0 being partitioned according to the subsystems and having all blocks zero except the (i,i) th, (i,j) th, (j,i) th and (j,j) th ones In this case, every subsystem pays an energetic price (for transmitting the value of its state to neighbors) for every adjacent edge Edge separable with interference: the map Q is still given by (6) but now each matrix P ij merely belongs to K n,n (g) This corresponds to a case where the energetic cost paid for communication over every link depends on all the other links present in the graph It is to capture this parasitic effect of edges on each other that we say that there is interference Non-separable: In this case, there is no edge-by-edge contribution to the information cost of a graph and Q(g) can be a full matrix for all g Such information costs can occur if subsystems not adjacent to a particular edge agree to subsidize it by reducing their energy In the next sections, we consider some classes of map Q and prove the existence of an optimal (or, again in the terminology of [9], efficient) graph g such that V (g ) V (g) g G N (7) 1 Hopefully, this should create no confusion as J g (Q) also corresponds to a value function in the optimal control theoretic sense of the term

3 in particular, V (g) = x T 0 P(g)x 0 where P(g) is the unique positive-definite solution of Riccati equation Fig 1 (a) (c) Some examples of graphs All but (b) are clique graphs The structure imposed by g corresponds to the minimal (in the sense of quadratic cost (3)) communication requirements needed to control the N subsystems described by (1) We also discuss the order defined by the value function V III MAIN RESULTS There is a natural partial order on G N We will write that g g whenever g is a subgraph of g, ie whenever E(g) E(g ) In this section, we set out to compare the values of any two graphs comparable for A Clique graphs and the efficient graph We start with the following proposition which is easily proven Proposition 1: (i) If g g, then for all Q > 0, J g (Q) J g(q) (ii) If Q Q then, for all g G N,J g (Q) J g(q ) We now introduce two general classes of map Q that complement those introduced in the previous sections Definition 1: We say that a map Q : G N S n is (i) non-decreasing if: g,g G N, g g Q(g) Q(g ) (ii) structure-compatible if: Q(g) K n,n (g) for all g G N When the map Q is non-decreasing, applying item (ii) of Proposition 1 to Q := Q(g) and Q := Q(g ), yields J g (Q(g)) V (g ) (8) Before we can go further and prove our first result on graph efficiency, we need to introduce the following concept borrowed from [7] Definition 2: A graph g G N is said to be a clique graph if each of its connected components is a clique ie, a complete subgraph Examples and counter-examples of clique graphs are given in Figure 1 As we will see, this type of graph is useful because the value can be readily computed, while determining the optimal structured controller for arbitrary structures is typically a hard problem (see eg, [6] and the references therein) Proposition 2: Let g G N be a clique graph and the map Q be structure-compatible Then V (g) = J (Q(g)) Hence, (b) (d) P(g)A + A T P(g) P(g)BB T P(g) + Q(g) = 0 (9) Proof: It is standard that the unstructured control law minimizing J(Q(g),) and the corresponding optimal value are given by Riccati equation (9) We want to show that, under our assumptions, this optimal control law in fact has structure g Since g is a clique graph, there exists a permutation matrix π such that π 1 A(g)π is block-diagonal and π 1 = π T This amounts to re-numbering the vertices of g so that vertices in the same clique have consecutive numbers and partitioning {1N} as C {1N} = I k, k=1 where C is the number of cliques in g and I k is the index set corresponding to clique k If we define the n n matrix Π by replacing every 1 entry in π by the identity matrix of proper dimensions, we obtain another permutation matrix which, by virtue of Q being structure-compatible, is such that Π T QΠ is also blockdiagonal: Π T QΠ = diag (Q k ) 1 k C We can also write Π T AΠ = diag k (A k ) and Π T B = diag k (B k ), with each block being itself block-diagonal and satisfying A k = diag i I k (A i ) ; B k = diag i I k (B i ) By assumption each pair (A i,b i ) is controllable and thus, according to [6], so is (A k,b k ) for each k Hence, the Riccati equation P(Π T AΠ) + (Π T AΠ) T P + Π T Q(g)Π P(Π T B)(Π T B) T P = 0 can be solved block-by-block and its unique positive-definite solution is clearly Π T P(g)Π It is also block-diagonal: Π T P(g)Π = diag k (P k ) Then the optimal control law is given by K(g) = B T P(g) = diag k (B T k )diag k(p k )Π T, which has structure g Proposition 2 and its proof are reminiscent of the results of [1], where it is shown that, for spatially invariant systems, the optimal controller is itself spatially invariant Here, the optimal controller has the same structure as the clique graph, when the map Q is structure-compatible This property allows us to compare the value of a clique graph to that of its supergraphs Theorem 1: Let Q be non-decreasing and structurecompatible and g be a clique graph Then V (g) V (g ) for all g G N such that g g In particular, the graph g characterized by E(g ) = is efficient, as defined in (7) In other words, the minimal control topology (for cost (3)) is fully decentralized

4 Proof: By Proposition 2, since g is a clique graph and Q is structure-compatible, V (g) = J (Q(g)) Also, since Q is non-decreasing, we can use relation (8) to write Jg (Q(g)) V (g ) Finally, by definition of the various minimization problems, we have V (g) = J (Q(g)) J g (Q(g)) V (g ) That the fully decentralized topology is minimal follows from the fact that g, as defined in the theorem, is a clique graph and, clearly, that g g for all g G N B Pricing edges The results of Theorem 1 are most easily understood when particularized to the case of an edge-separable map Q add without interference, as defined in Section II For Q add to be structure-compatible, the matrix Q 0 must be blockdiagonal (each block being of size n i and corresponding to a subsystem), while matrices P ij do not need to satisfy any additional constraint In this context, Theorem 1 says that, given that communication has a cost (no matter how small) and the original performance criterion is block-diagonal, it is detrimental to include any edge in the controller s interconnection topology We now give sufficient conditions for the addition of edges to any graph to be detrimental Theorem 2: Let g g Assume that the map Q is nondecreasing If there exist K K m,n (g ) and P > 0 such that 0 = (A BK) T P + P(A BK) + Q(g ) + K T K Q(g) Q(g ) + (K B T P) T (K B T P) (10a) (10b) then V (g) V (g ) ie, it is detrimental (in the sense of cost (3)) to add the edges in E(g)\E(g ) to a controller with original topology specified by graph g In addition, if g is a clique graph and Q is structurecompatible, then Q(g) Q(g ) P V (g) V (g ), (11) where P := ( K(g ) B T P(g )) T ( K(g ) B T P(g )) and K(g ), P(g ) are the optimal structured controller and cost matrix (ie the solution of (10a) for K = K(g )) for g, respectively Proof: Assume P > 0 satisfies equation (10a) Then we claim that J g (Q(g ),u) = x T 0 Px 0 for the structured control law u = Kx Indeed, since Q(g ) + K T K > 0, matrix (A BK) is Hurwitz Also, from (10a), we see that d dt xt (t)px(t) = x T (t)q(g )x(t) + u T (t)u(t) for every trajectory x of the the closed-loop system ẋ = (A BK)x Then, integrating this equality between t = 0 and t = and using the fact that lim x(t) = 0 gives the t desired result Also, from (10a), we see that P satisfies A T P + PA PBB T P + (Q(g ) + (K B T P) T (K B T P)) = 0 Combining these two facts, we see that J g (Q(g ),u) = J (Q(g ) + (K B T P) T (K B T P)) If inequality (10b) holds, then using item (ii) of Proposition 1 for the complete graph, we get V (g ) J g (Q(g ),u) J (Q(g)) V (g) Finally, if g is a clique graph then V (g) = J (Q(g)) If (11) holds, we can use the same reasoning as before to write V (g) J (Q(g ) + P) = J g (Q(g ),ū) with ū = K(g )x ie, V (g) V (g ) by definition of K(g ) Conditions (10) can be used to design a map Q to enforce some desired control topology For example, imagine a situation where the map Q is chosen by one institution (the price-designer) while the controller is synthesized by another one (the network-builder) and that a topology, g, has been agreed on and implemented Then, the price designer can ensure that no new edge will be built by choosing a stabilizing control gain K K(g ), solving Lyapunov equation (10a) and picking Q such that (10b) holds On the other hand, the network-builder is only given the map Q If it wants to use conditions (10) to determine whether it is to its advantage to add new edges to a pre-existing control topology (or, more aptly, to find a certificate that it is detrimental to do so) it has to solve equations (10) for both P and K Even after using the Schur complement formula on inequality (10b) and rewriting (10a) as two matrix inequalities, this is still a hard task to perform, since one then has to solve a set of bilinear matrix inequalities which, in general, is NP-hard [18] We now give another sufficient condition which, at least for some graphs, allows us to answer the network-builder question in a tractable way Theorem 3: Let g be a complete graph and g be a subgraph of g Also assume that the map Q is structurecompatible Then V (g) > V (g ) if there exists K K m,n (g ) such that the following linear matrix inequality (LMI) is satisfied: [ (A BK) T P(g) + P(g)(A BK) + Q(g ) K T K I < 0, (12) where P(g) is the positive-definite solution of Riccati equation (9) Proof: Let K K m,n (g ) satisfy LMI (12) Using Schur complement, it also satisfies (A BK) T ( P(g))+( P(g))(A BK) > Q(g )+K T K (13) Then, using similar arguments as in the proof on Theorem 2, we obtain x T (0)P(g)x(0) > J g (Q(g ),u) To finish the proof, note that, since g is complete, it is a clique graph and thus the left-hand side of the previous inequality is just V (g) according to Proposition 2 while, by definition, V (g ) J g (Q(g ),u) ]

5 Note that Theorem 3 is different from Theorem 2 in that it allows us to prove that it is detrimental to add edges to a graph g (when the resulting graph is complete) knowing only the map Q: one just has to solve Riccati equation (9) to determine P(g) and then solve an LMI in K C An example The results of the previous sections allow us to give a complete picture for the case N = 3 The set G 3 of all graphs over three vertices is depicted in Figure 2 in the form of a lattice corresponding to the partial order We consider the case of an edge-separable map Q with no interference, as defined in Section II, and with Q 0 = I Clearly, this map is both non-decreasing and structure compatible Thanks to Theorem 1, we thus immediately know that V (g 0 ) V (g) for all g, V (g 4 ) V (g 2 ),V (g 3 ), V (g 5 ) V (g 1 ),V (g 3 ), V (g 6 ) V (g 1 ),V (g 2 ), V (f) V (g 0 ),V (g 1 ),V (g 2 ),V (g 3 ), but we cannot a priori compare the values of f with that of g 4, g 5 or g 6 However, using a gradient descent algorithm similar to that introduced in [6] to approximate the optimal controllers and cost matrix for the graphs g 4, g 5, and g 6, we can resort to Theorem 2 to determine the corresponding critical prices Consider the simple case where all subsystems have a single state, and P 12 = A = , P 23 = ; B = I We can obtain an approximate value of V (g 5 ) and find that ( K(g 5 ) P(g 5 )) T ( K(g 5 ) P(g 5 )) = after convergence of the algorithm of [6] Then, using the same computational tool, one can verify that taking P 13 = ( K(g 5 ) P(g 5 )) T ( K(g 5 ) P(g 5 )), clique graph complete graph g 4 g 5 g 6 g 1 g 2 g 3 g 0 f clique graph Fig 2 The poset (G 3, ) The continuous lines indicate that for the map Q given in the text, the bottom graph has value less than the top graph The dashed lines indicate transition with possible critical prices that can be computed using Theorem 2 See text for details always yields V (f) < V (g 5 ), while P 13 = gives V (f) > V (g 5 ) ( K(g 5 ) P(g 5 )) T ( K(g 5 ) P(g 5 )) IV CONCLUSION AND PERSPECTIVES This paper is only a first attempt at studying the influence of interconnection topology on performance in distributed control design problems The main virtue of the proposed criterion is that it allowed us to make rigorous statements regarding the optimal topology, thus complementing the more heuristic claims of [2] and [17] Even within the realm of this restricted model, many unanswered questions remain For example, it would be nice to be able to compare the values of any two graphs and not only of those that are comparable for the partial order Likewise, one may ask whether it is possible to derive bounds similar to those of Theorem 2 that are tractable and/or explicit However, we believe that the most interesting questions raised by this approach are of a different nature While our model was inspired by the framework of Network Formation Theory, our results do not really fit into it because (in keeping with classical control design practice) we have assumed that a single institution (the network builder of Section III) could decide the full information pattern of the controller when searching for the efficient communication graph Another approach, which is closer to the spirit of Games over Networks and which we are currently investigating, is to assume that each subsystem can decide to create new communication links independently from the others, based on its own reward derived from a particular network or, in the cooperative case (see eg, [13], [16]), pay-off allocation rule The efficient topology would then be obtained as a Nash equilibrium or core of the corresponding game and the optimal controller could thus be constructed in a fully distributed fashion

6 ACKNOWLEDGMENTS Discussions with Prof Anders Rantzer and Tudor Stoenescu, on this and related topics, are gratefully acknowledged REFERENCES [1] B Bamieh, F Paganini, M A Dahleh Distributed control of spatially invariant systems IEEE Transactions on Automatic Control, vol 47, no 7, pp , July 2002 [2] G M Belanger, S Ananyev, J L Speyer, D F Chichka, and J R Carpenter Decentralized Control of Satellite Clusters Under Limited Communication In AIAA Guidance, Navigation and Control Conference, August 2004 AIAA paper: [3] Bellman, R (editor) Special issue on Large systems IEEE Transactions on Automatic Control, vol 19, no 5, pp , Oct 1974 [4] V Blondel, J Tsitsiklis NP-hardness of some linear control design problems SIAM Journal of Control and Optimization, 35:6, pp , 1997 [5] W B Dunbar and R M Murray Receding horizon control of multivehicle formations: a distributed implementation In Proceedings of the IEEE Conference on Decision and Control, Paradise Island, Bahamas, pp , 2004 [6] V Gupta, B Hassibi, and R M Murray A suboptimal algorithm to synthesize control laws for a network of dynamic agents International Journal of Control, vol 78, no 16, pp , Nov 2005 [7] F Harary, Graph Theory, Addison-Wesley, 1994 [8] Y C Ho and K Chu Team decision theory and information structures in optimal control problems Part I IEEE Transactions on Automatic Control, vol 17, no 1, pp 15-22, Feb 1972 [9] M O Jackson A Survey of Models of Network Formation: Stability and Efficiency In Group Formation in Economics: Networks, Clubs, and Coalitions by G Demange and M Wooders (eds), Cambridge University Press, 2004 [10] C Langbort, R S Chandra and R D Andrea Distributed Control Design for Systems Interconnected over an Arbitrary Graph IEEE Transactions on Automatic Control, vol 49, no 9, pp , Sept 2004 [11] X Liu and A J Goldsmith Wireless Network Design for Distributed Control In Proceedings of IEEE Conference on Decision and Control, Paradise Island, Bahamas, pp , 2004 [12] J Marschak and R Radner Economic Theory of Teams Cowles Foundation Monograph 22, Yale University Press, 1972 [13] R Myerson Graphs and Cooperation in Games Mathematics of Operations Research, vol 2, pp , 1977 [14] M Rotkowitz and S Lall Decentralized Control Information Structures Preserved Under Feedback In Proceedings of IEEE Conference on Decision and Control, Las Vegas, NV, Dec 2002, vol 1, pp [15] N Sandell and M Athans Solution of some nonclassical LQG stochastic decision problems IEEE Transactions on Automatic Control vol 19, no 2, pp , Apr 1974 [16] M Slikker and A van den Nouweland Social and Economic Networks in Cooperative Game Theory, Kluwer Academic Publishers, Boston, 2001 [17] R Smith and FHadaegh Closed-loop Dynamics of Cooperative Vehicle Formations with Parallel Estimators and Communication Submitted to IEEE Transactions on Automatic Control, 2005 [18] O Toker and H Özbay On the NP-hardness of solving bilinear matrix inequalities and simultaneous stabilizationwith static output feedback In Proceedings of the American Conference on Control, Seattle, WA, pp , 1995 [19] S Wang and E J Davison On the stabilization of decentralized control systems IEEE Transactions on Automatic Control, vol 18, no 5, pp , Oct 1973 [20] H S Witsenhausen A Counterexample in Stochastic Optimum Control SIAM Journal of Control, 6(1), 1968, pp