Optimal Control of Spatially Distribute Systems Naer Motee an Ali Jababaie Abstract In this paper, we stuy the structural properties of optimal control of spatially istribute systems. Such systems consist of an infinite collection of possibly heterogeneous linear control systems that are spatially interconnecte via certain istant epenent coupling functions over arbitrary graphs. The key iea of the paper is the introuction of a special class of operators calle spatially ecaying (SD) operators. Such operators can be thought of as generalization of translation invariant operators use in the stuy of spatially invariant systems as well as operators efine base on nearest neighbor couplings in networke ynamic systems. We stuy the structural properties of infinite-horizon linear quaratic optimal controllers for such systems by analyzing the spatial structure of the solution to the corresponing operator Lyapunov an Riccati equations. We prove that the kernel of the optimal feeback of each subsystem ecays in the spatial omain at a rate proportional to the inverse of the corresponing coupling function of the system. When the coupling is nearest neighbor base or exponentially ecaying, we show that the coupling in the optimal controls ecays exponentially in space. In the case when the systems are couple using an algebraic function, it is proven that the coupling in the optimal controls ecays algebraically. Our theoretical results are verifie by numerical simulations. I. INTRODUCTION Analysis an synthesis of istribute coorination an control algorithms for networke ynamic systems has become a vibrant part of control theory research. From consensus an agreement problems to formation control an sensing an coverage problems, researchers have been intereste in control algorithms that are spatially istribute an esigne to achieve a global objective such as consensus or coverage using only local interactions [1] [9]. Other examples of research on istribute ynamic systems inclue istribute optimization-base control unmanne aerial vehicles [1], automate highway systems [11]- [12], cross-irectional control systems for inustrial paper machines [13], large segmente telescopes [14], power istribution systems [15], an arrays of microcantilevers for massively parallel ata storage [16] only to name a few of these applications. In aition to the above results, several authors have stuie the problem of optimal control of certain classes of spatially istribute systems with symmetries in their spatial structure (cf. Figures 1 an 2). In [17], Bamieh et al. use spatial Fourier transforms an operator theory to stuy optimal control of linear spatially invariant systems with stanar H 2 (LQ), an H criteria. It was shown that such problems can be tackle by solving a parameterize family of finite-imensional N. Motee an A. Jababaie are with the Department of Electrical an Systems Engineering an GRASP Laboratory, University of Pennsylvania, 2 South 33r Street, Philaelphia PA 1914. {motee,jababai}@seas.upenn.eu This work is supporte in parts by the following grants: ONR/YIP N14-4-1-467, NSF-ECS-347285, an ARO MURI W911NF-5-1- 381 problems in Fourier omain. Furthermore, the authors show that the resulting optimal controllers have an inherent spatial locality similar to the unerlying system. In [18], the authors evelope conitions for wellposeness, stability, an performance of spatially interconnecte systems whose moel consist of homogeneous units on a iscrete group (e.g. over a one-imensional or twoimensional lattice or ring), in terms of linear matrix inequalities (LMI). These results were later extene to systems with certain types of bounary conitions [19]. In [2], the applicability of results of [18] is extene to a larger class of interconnection topologies with arbitrary iscrete symmetry groups. Another relate result is reporte in [21], where the authors use a scale small gain theorem, to give a unifie interpretation to the analysis results of [2]. Another relate result is that of [22], where the problem of istribute controller esign with a funnel causality constraint is shown to be a convex problem, provie that the plant has a similar funnelcausality structure, an the propagation spee in the controller is at least as fast as those in the plant. Another relevant result is ue to the authors in [23] in which a ecentralize control approach to spatially invariant systems is stuie an the interactions between agents are moele as isturbances satisfying certain magnitue bouns. In [24], the backstepping approach is utilize to aress stabilization, regulation, an asymptotic tracking of nominal systems an systems with parametric uncertainties on lattices. It is also shown that the esigne controllers have the same architecture as the original plant. Another much oler but relate work on this subject was reporte in [25] where homogeneous interconnecte systems are stuie using Z-transform analysis. Furthermore, it is shown that many homogeneous large-scale systems can be reasonably approximate by an infinite number of couple ientical subsystems. The authors in [26] employ tools from issipativity theory to synthesize optimal controllers for spatially interconnecte systems with non-ientical units over an arbitrary graph. Another interesting work in this area is reporte in [27] where the authors use operator theoretic tools, motivate by results of [28] to analyze time-varying systems, an esign optimal controllers for heterogeneous systems which are not shift invariant with respect to spatial or temporal variables. In [29] an [13], the problem of synthesizing an implementing a istribute controller with practical/inustrial applications is consiere. Another recent work in this area is reporte in [3] in which the authors introuce the notion of quaratic invariance for a constraint set (e.g. sparsity constraints on communication structure of plant an controller). Using this notion, the authors show that the problem of constructing optimal controllers with certain sparsity patterns on the information structure can be cast as a convex optimization problem. In all of these
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 2 Fig. 1. Spatial invariance on a hexagonal array. papers, except for [17], [3], a synthesis-base approach is taken to evelop a control esign metho which yiels a istribute controller with possibly the same architecture as the unerlying plant. This paper is very close in spirit to [17]. The objective of this paper is to analyze the spatial structure of infinite horizon optimal controllers of spatially istribute systems. Here, we exten the results of [17] to heterogeneous systems with arbitrary spatial structure an show that quaratically optimal controllers inherit the same spatial structure as the original plant. The key point of eparture from [17] is that the systems consiere in this work are not spatially invariant an the corresponing operators are not translation invariant either. The spatial structures stuie in [17] are Locally Compact Abelian (LCA) groups [31] such as (Z, +) an (Z n, ). As a result, the group operation naturally inuces a translation operator for functions efine on the group. However, when the ynamics of iniviual subsystems are not ientical an the spatial structure oes not necessarily enjoy the symmetries of LCA groups, stanar tools such as Fourier analysis cannot be use to analyze the system. To aress this issue, a new class of linear operators, calle spatially ecaying (SD) operators, are introuce that are natural extension of linear translation invariant operators. Roughly speaking an operator is SD with respect to an inuce norm if a certain auxiliary operator forme by blockwise exponential (or algebraic) inflation of the operator remains boune. It is shown that such operators exhibit a localize behavior in spatial omain, i.e., the norm of blocks in the matrix representation of the operator ecay exponentially or algebraically in space. A trivial example of an SD operator are those representing nearest neighbor coupling such as the graph Laplacian, which has receive significant attention in cooperative control literature recently [1], [5], [6], [8]. Another important subclass inclues any boune translation invariant operator. It turns out that the coupling between subsystems in many well-known cooperative control an networke control problems can be characterize by an SD operator. A linear control system is calle spatially ecaying if the operators in its state-space representation are SD. It is shown that the space of SD operators S(C ) is a norme vector space with respect to a specific operator-norm which is not inuce an is enote by, b. Furthermore such operators equippe with the norm form a Banach algebra. Using this result, we prove certain closure properties of this space with respect to convergent sequences on l p. This will then enable us to prove that the unique solution of Lyapunov an algebraic Riccati equations (ARE) corresponing to SD system are Fig. 2. Spatially invariant system over a two-imensional lattice G = Z Z inee SD themselves. As a result, the corresponing optimal controllers are SD an spatially localize, meaning that in the optimal controller, the gain of subsystems that are farther away from a given subsystem ecays in space an the resulting controller is inherently localize. Specifically, we show that when the coupling function between subsystems is exponential, the size of the kernel of corresponing optimal controller ecays exponentially, an when the coupling is algebraic, optimal controllers ecay algebraically in space. For the specific case of nearest neighbor coupling, the ecay is shown to be exponential. The machinery evelope in this paper can be use to analyze the spatial structure of a broaer range of optimal control problems such as constraine, finite horizon control or Moel Preictive Control as well as. This problem has been analyze in etail in [32], an [33]. This paper is organize as follows. We introuce the notation an the basic concepts use throughout the paper in Section II. The optimal control problem for spatially istribute linear systems an the motivation of the paper are presente in Section III. The concept of spatially ecaying operators is introuce in Section V. Three important type of SD operators are stuie in etails in Section VI. Results of Section VII which is about the properties of SD operators are utilize in Section VIII to show that the solutions of Lyapunov equations an Algebraic Riccati Equations ARE inherits spatial locality. Simulation results are inclue in Section I, in which we emonstrate the ecay of spatial coupling in the optimal solution by consiering interconnection of a large network of interconnecte units on an arbitrary connecte graph. Finally, our concluing remarks are presente in Section. II. PRELIMINARIES The notation use in this paper is fairly stanar. R enotes the set of real numbers, R + the set of nonnegative real numbers, C the set of complex numbers, an S 1 the unit circle in C. Consier the Hilbert spaces H i equippe with inner proucts, Hi for i G where G is an inex set. We refer to G as the spatial omain (see Fig. 1 an 2 for two specific examples). The inner prouct on each Hilbert space H i inuces the norm x i Hi =È x i, x i Hi for all x i H i. Whenever it is clear from the context, all inuce norms of linear maps between two Hilbert spaces H i an H j are simply enote by.. The Banach space l p (G) for 1 p < is efine to be the set of all sequences x = (x i ) in which
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 3 375 x i H i satisfying matrix representations 264. x i p H i <.. Q enowe with the norm x p := x i p H i!1 p. The Banach space l (G) enotes the set of all boune sequences enowe with the norm x := sup x i Hi. Throughout the paper, we will use the shorthan notation l p for l p (G). The space l 2 is a Hilbert space with inner prouct x, y := x i, y i Hi for all x, y l 2. An operator Q : l p l q for 1 p, q is boune if it has a finite inuce norm, i.e., the following quantity Q lp l q := sup x p=1 Qx q (1) is boune. The ientity operator is enote by I. The set of all boune linear operators of l p into l q for some 1 p, q is enote by L(l p, l q ). The space L(l p, l q ) equippe with norm (1) is a Banach space (cf. [34]), an in the case where the initial an target spaces are both l p, we use the notation L(l p ). An operator Q L(l p ) has an algebraic inverse if it has an inverse Q 1 in L(l p ) [34]: QQ 1 = Q 1 Q = I. The ajoint operator of Q L(l 2 ) is the operator Q in L(l 2 ) such that Qx, y = x, Q y for all x, y l 2. An operator Q is self-ajoint if Q = Q. An operator Q L(l 2 ) is positive efinite, shown as Q, if there exists a number α > such that x, Qx > α x 2 2 for all nonzero x l 2. The set of all functions from A R into R is a vector space F over R. For f 1, f 2 F, the notation f 1 f 2 will be use to mean the pointwise inequality f 1 (s) f 2 (s) for all s A. A family of seminorms on F is efine as {. T T R + } in which f T := sup f(s) st for all f F. The topology generate by all open. T -balls is calle the topology generate by the family of seminorms an is enote by. T -topology. Continuity of a function in this topology is equivalent to continuity in every seminorm in the family. Remark 1: Although the results of section III is set up in a general framework, in this paper we are intereste in linear operators Q : l p l q for some 1 p, q which have [Q] ki... where the block element [Q] ki is a linear map from H i into H k. Furthermore, it is assume that H i = C n where n > is an integer. III. OPTIMAL CONTROL OF SPATIALLY DISTRIBUTED SYSTEMS We begin by consiering a continuous-time linear moel for spatially istribute systems over a iscrete spatial omain G escribe by ψ(t) t = (Aψ) (t) + (Bu) (t) (2) y(t) = (Cψ) (t) + (Du) (t) (3) with the initial conition ψ() = ψ. All signals are assume to be in L 2 ([, ); l 2 ) space: at each time instant t [, ), signals ψ(t), u(t), y(t) are assume to be in l 2. The statespace operators A, B, C, D are assume to be time-invariant linear operators from l 2 to itself. The following assumption guarantees existence an uniqueness of classical solutions of the system given by (2)-(3) (cf. Chapter 3 of [35] for more etails). Assumption 1: The semigroup generate by A is strongly continuous on l 2. The following is an example of a spatially istribute system on G = Z. Example 1: Consier the general one-imensional heat equation for a bi-infinite bar [36] ψ(x, t) = t x c(x) x ψ(x, t) +b(x)u(x, t) where x is the spatial inepenent variable, t is the temporal inepenent variable, ψ(x, t) is the temperature of the bar, an u(x, t) is a istribute heat source. The thermal conuctivity c is only a function of x an is ifferentiable with respect to x. The bounary conitions are assume to be ψ(, t) = ψ(, t) =. By inserting finite ifference approximation for the spatial partial erivatives, the following continuoustime, iscrete-space moel can be obtaine: t ψ(x k, t) = c k 1, t) ψ(x k, t) (x k ) ψ(x δ k 1, t) 2ψ(x k, t)ψ(x k+1, t) + c(x k ) ψ(x δ 2 + b(x k )u(x k, t) where c (x) = xc(x). The iscretization is performe with equal spacing δ = x k x k 1 of the points x k such that there is an integer number of points in space. Hence, after iscretization the spatial omain is G = Z. This moel can be represente as ψ(t) = (Aψ) (t) + (Bu) (t) t
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 4 in which the infinite-tuples ψ(t) = (ψ(x k, t)) k G an u(t) = (u(x k, t)) k G are 8><>: the state an control input variables of the infinite-imensional system an the block elements of the state-space operators A an B are efine as follows for every c (x k )δ+c(x k ) δ, i = k 1 2 [A] ki = c (x k )δ+2c(x k ) δ, i = k 2 c(x k ) δ, i = k + 1 2, otherwise an [B] ki = b(x k ), i = k, otherwise for all k G. One can show that A is a boune operator on l 2 an as a result, it generates a uniformly continuous semigroup. Furthermore the generate semigroup is strongly continuous on l 2. In the next section, we stuy the exponential stability problem for autonomous systems of the form (4) as well as linear quaratic optimal control problems for systems escribe by (2)-(3). The focus of this paper woul be on LQR problems, however, results are vali for general H 2 an H optimal control problems as well. The key ingreient of the result is to prove spatial locality of solutions of Lyapunov an Riccati equations. A. Exponential Stability Consier the following autonomous system over G ψ(t) = (Aψ) (t) (4) t with initial conition ψ() = ψ. Suppose that A generates a strongly continuous C -semingroup on l 2, enote by T (t). Exponential stability can be efine as follows. Definition 1: The system (4) is exponentially stable if for some M, α >. T (t) l2 l 2 Me αt for t Similar to the finite imensional case, one can efine a similar Lyapunov equation in an operator theoretic framework for infinite-imensional systems. The following theorem from [35] is stanar an provies such an extension. Theorem 1: Let A be the infinitesimal generator of the C - semigroup T (t) on l 2 an Q a positive efinite operator. Then T (t) is exponentially stable if an only if the Lyapunov equation Aφ, Pφ + Pφ, Aφ + φ, Qφ = (5) for all φ D(A), has a positive efinite solution P L(l 2 ). Solving the Lyapunov operator equation (5) can be a teious task in general. However, the complexity of the problem will reuce significantly if the unerlying system is spatially invariant with respect to G (cf. Section III.B of [17]). This will e iscusse in more etail later on. B. LQR control of infinite imensional systems We now review the basics of linear quaratic regulator theory for infinite-imensional systems. While the main results of this paper are proven for LQ optimal controllers, similar results can be proven for H an H 2 problems. In general, the solutions to these problems can be formulate in terms of two operator AREs. Such problems have been aresse in the literature for general classes of istribute parameter systems [35], [37]. A complete an elegant analysis for the spatially invariant case can be foun in [17]. From now on, we will only focus on the structure of the solution to LQR problems for systems escribe by (2)-(3). Such problems have been well stuie in the istribute parameter system literature [35], [37]. Similar to the finite-imensional case, optimal solutions to infinite-imensional LQR can be formulate in terms of an operator Riccati equation. Consier the quaratic cost functional given by J =Z ψ(t), Qψ(t) + u(t), Ru(t) t. (6) The system (2)-(3) with cost (6) is sai to be optimizable if for every initial conition ψ() = ψ l 2, there exists an input function u L 2 ([, ); l 2 ) such that the value of (6) is finite [35]. Note that if (A, B) is exponentially stabilizable, then the system (2)-(3) is optimizable. The following is a stanar result from [35]. Theorem 2: Let operators Q an R be in L(l 2 ). If the system (2)-(3) with cost functional (6) is optimizable an (A, Q 1/2 ) is exponentially etectable, then there exists a unique nonnegative, self-ajoint operator P L(l 2 ) satisfying the ARE ϕ, PAφ + PAϕ, φ + ϕ, Qφ B Pϕ, R 1 B Pφ = (7) for all ϕ, φ D(A) such that A BR 1 B P generates an exponentially stable C -semigroup. Moreover, the optimal control ũ L 2 ([, ); l 2 ) is given by the feeback law where ψ is the solution of ũ(t) = R 1 B P ψ(t) t ψ(t) = (A BR 1 B P) ψ(t) (8) with initial conition ψ. Similar to the case of operator Lyapunov equations, Equation (7) is ifficult to solve in general. When the system (2)-(3) is spatially invariant, equation (7) reuces to a parameterize family of finite-imensional LQR problems [17]. As mentione in the introuction, the main objective of this paper is to analyze the spatial structure of the solutions of operator equations (5) an (7) rather than solving them explicitly. In section VIII, it will be shown that the unique solution of these operator equations (uner the appropriate stanar assumptions) have an inherent spatial locality property.
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 5 IV. SPATIALLY INVARIANT SYSTEMS In orer to motivate our results on structure of optimal control for general spatially istribute systems, we first consier the important subclass of spatially invariant systems on iscrete groups (Z, +). Note that this problem has been stuie extensively in [17] with emphasis on continuous group (R, +). We will mention these results an moify them when necessary for the iscrete group G = Z. In what follows, the Banach space l p is efine exactly the same as in Section II with an aitional assumption that all u i in the infinite-tuples (u i ) i Z l p belong to the same Hilbert space C n. We begin by introucing the unit translation operator to the right with respect to the group operation + as follows Tu = T(..., u i, u i+1,... ) = (..., u i 1, u i,... ). One can verify that T lp l p = 1 for all 1 p. Higher orer translation operators can be efine iteratively by T s = T s 1 T for all s Z. We are now reay to efine translation invariant operators. Definition 2: Operator Q : D(Q) l p with omain D(Q) l p is translation invariant if it commutes with every translation operator T s : D(Q) D(Q), i.e., T s Q = QT s for all s Z. It can be shown that all linear translation invariant operators on l p = can be characterize by forming linear combinations of higher orer translation operators of the form Q(T) Q k T k (9) k Z with Q k C n n. Definition 3: The system escribe by equations (2)-(3) is calle spatially invariant if the state-space operators A, B, C, D are translation invariant. For every x l 2 =, the iscrete Fourier transform is efine by ˆx(z) x k z k k Z where z S 1. Using this efinition, one can compute the iscrete Fourier transform of a translation invariant operator. We will assume that the Fourier transform of all operators are continuous. A translation invariant operator Q is boune on l 2 [31] if an only if Q l2 l 2 = sup z S 1 ˆQ(z) <. (1) It can be shown (see Theorem?? in the appenix) that for translation invariant operators bouneness on l 2 implies bouneness on all l p spaces for 1 p. The following ecay result for iscrete group Z is similar to that of Theorem 7.4.2 of [38] for continuous group R (see also Theorem 5 of [17] for the continuous space version). Theorem 3: Let Q be efine by (9) on Z with iscrete Fourier transform ˆQ(z). If Q L(l 2 ), then the coefficients of operator Q ecay exponentially in the spatial omain, i.e., for S 1 Im(z) 1 r 1 + r Re(z) Fig. 3. Analytic continuation to annulus Ω when G = Z. all k Z Q k α e β k (11) for some α > an < β < ln(1+r) where r is the istance of the closest pole of ˆQ(z) to S 1. Proof: Accoring to (1) if Q L(l 2 ), then ˆQ(z) has no pole on S 1, an it has analytic continuation to some annulus Ω = {z C : 1 r < z < 1 + r, r > } (12) where r is the istance of the closest pole of ˆQ(z) to S 1. Now consier the moifie operator Q which is efine by Q k = Q k ζ k. One can see that Q is also a translation invariant operator. From (1), it follows that ˆQ(ζe iω ) < for all 1 r < ζ < 1 + r. Therefore, by using the inequality Q k Q l2 l 2 for all k Z, the ecay result (11) can be obtaine immeiately. As shown in [17], the solutions of operator ARE for a spatially invariant system reuces to the following parameterize equation  ˆP + ˆP ˆP ˆB ˆR 1 ˆB ˆP + ˆQ = (13) which is evaluate on S 1. The spatial frequency omain ineterminant z has been roppe from the above equation for notational simplicity. Assuming that all conitions of theorem 2 are satisfie, equation (13) has a unique boune solution P on l 2. Furthermore, if the Fourier transform of all operators A, B, Q, R have analytic continuation to some annulus aroun the unit circle, a similar argument as in Section V.B.1 of [17] can be use to show that the Fourier transform of P also has analytic continuation to an annulus aroun the unit circle. This, in combination with Theorem 3, guarantees that the coefficients of the translation invariant operator P, ecay exponentially in the spatial omain, i.e., P k α e β k (14) for some α, β >. Note that the spatial ecay of the solution in (14) is ientical to that of [17] for continuous group R with the minor ifference that aitional assumptions on growth
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 6 bouns for ˆP(z) are not require (See appenix B of [17] for more etails). This is ue to the fact that the annulus is a compact set in C, an ˆP(z) is a continuous function (in the case of a continuous group, a strip aroun the imaginary axis is not boune). Therefore, the extreme points are attaine on the set. In summary, given a boune translation invariant operator on l 2, analytic continuity of its Fourier transform guarantees spatial locality of the operator by guaranteeing that the operator ecays exponentially in space. Unfortunately, the applicability of this result is limite to systems that are highly symmetric (such as ientical ynamics on a lattice). The main question that we are trying to answer here is whether these concepts can be extene to a larger class of operators which are not necessarily spatially invariant. This question is answere in a rigorous fashion in the next section. It turns out that the notion of spatial locality can be extene from translation invariant operators to a larger class of linear operators. This requires extening the notion of spatial ecay in a natural way from linear translation invariant operators to a larger class of linear operators calle spatially ecaying or SD for short V. SPATIALLY DECAYING OPERATORS The key ifficulty in extening the results of previous section is that the notion of spatial invariance was critical in being able to use Fourier methos which greatly simplifie the analysis. Simply put, if we replace space with time, we get a more familiar analogue of this problem: Fourier methos can not be use irectly for analysis of linear time-varying systems. The key in extening the results of the previous section to systems that are not spatially invariant is to somehow exten the notion of analytic continuity. Consier the boune translation invariant operator Q of form (9) with iscrete Fourier transform ˆQ(z) which has analytic continuation to some annulus Ω aroun the unit circle S 1. Suppose that Γ is a circle with raius ζ > 1 an strictly lies insie Ω. By analytic continuity, it follows that ˆQ(z) < for all z Γ. Now consier the following inequality ˆQ(z) = Q k z k k Z Q k ζ k e iωk k Z Q k ζ k. (15) = k Z Applying the result (11) to (15), it can be shown (see Section VI-B for etails) that the quantity in the right han sie of inequality (15) is boune. This shows that Q k ζ k < (16) k Z if an only if ˆQ(z) has analytic continuity on some annulus aroun the unit circle which strictly contains Γ. Consier the moifie linear operator Q(ζ) efine by [ Q(ζ)] ki = Q k i ζ k i. One can see that the moifie operator Q(ζ) is also translation invariant. If conition (16) hols, from (1) an (15) we see that Q(ζ) l2 l 2 is boune. Therefore, by applying theorem?? we have the following result. Proposition 1: The ˆQ(z) has analytic continuation to some annulus Ω aroun the unit circle which strictly contains the circle Γ of raius ζ > 1 if an only if Q(ζ) L(l p ). The above proposition suggests that analytic continuity is equivalent to bouneeness of an auxiliary operator Q(ζ), which is the exponentially weighte version of the original operator. In the following, we will generalize this iea to a larger class of linear operators by first forming an auxiliary weighte operator an imposing bouneness of the moifie operator on l p. Before oing so, we nee to review the notion of a istance function. Definition 4: A istance function on a iscrete topology with a set of noes G is efine as a single-value, nonnegative, real function is(k, i) efine for all k, i, j G which has the following properties: (1) is(k, i) = iff k = i. (2) is(k, i) = is(i, k). (3) is(k, i) is(k, j) + is(j, i). Next, we will efine the notion of a coupling characteristic function which will then be use as a weight function in the auxiliary operator. Definition 5: A nonecreasing continuous function χ : R + [1, ) is calle a coupling characteristic function if χ() = 1 an χ(s + t) χ(s) χ(t) for all s, t R +. The constant coupling characteristic function with unit value everywhere is enote by 1. In orer to be able to characterize rates of ecay we efine a one-parameter family of coupling characteristic functions as follows Definition 6: A one-parameter family of coupling characteristic functions C is efine to be the set of all characteristic functions χ α for α R + such that (i) χ = 1. (ii) For all χ α, χ β C with α < β, relation χ α χ β (iii) hols. χ α is a continuous function of α in. T -topology, i.e., for every T > an any given ε >, there exists δ > such that for all α β < δ. χ α χ β T < ε A simple example of such a one-parameter family is the family of exponential functions e α x for x, α R +. As one can see, this family satisfies the above efinitions. Using this efinition, we can now formally efine a spatially ecaying (SD) operator. Definition 7: Suppose that a istance function is(.,.) an a
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 7 one-parameter family of parameterize coupling characteristic functions C are given. A linear operator Q L(l p ) is SD with respect to C if there exists b > such that the auxiliary operator Q(α), efine block-wise as [ Q(α)] ki = [Q] ki χ α (is(k, i)) is boune on l p for all α < b. The number b is referre to as the ecay margin. In general, etermining the bouneness of the auxiliary operator epens on the choice of p. When p = bouneness can be easily characterize. The following result gives us a simple sufficient conition in terms of l norm. Lemma 1: A linear operator Q L(l ) is SD with respect to the one-parameter family of coupling characteristic functions C on l p for all 1 p if there exists b > such that the following hols sup for all α < b. [Q] ki χ α (is(k, i)) < (17) Proof: First, we will show that the auxiliary operator Q(α) is simultaneously in L(l 1 ) an L(l ). For a fixe α [1, b), Q(α) L(l1 ) L(l ) if the following quantities an Q(α) l l sup Q(α) l1 l 1 sup [Q] ki χ α (is(k, i)) [Q] ki χ α (is(k, i)) are boune. Finally, using the Riesz-Thorin theorem, it follows that Q is also boune on all intermeiate spaces l p where 1 p. Examples of SD operators in L(l ) appear naturally in many applications. For every operator efine on a finite graph with a finite number of noes, conition (17) always hols. Intuitively, we may interpret the norm of each block element [Q] ki as the coupling strength between subsystems k an i. Given the one-parameter family of coupling characteristic functions C, fix a value for α (, b). If noes k an i are taken far away from each other, the coupling strength between them will weaken so as to make the quantity in (17) boune. Inee, the ecay will be proportional to the inverse of the coupling characteristic function χ α. For example, if the coupling characteristic function is chosen to be exponential, the coupling strength will ecay exponentially. For an infinite graph, if we fix a noe k an move on the graph away from noe k, the coupling strength ecays proportional to the inverse of the coupling characteristic function χ α so that relation (17) hols. The notion of an SD operator will be key in proving spatial locality of optimal controllers. Remark 2: Throughout the rest of the paper, by SD we mean SD in l. One can easily see that all spatially invariant systems are inee SD with respect to exponential coupling characteristic functions. We finish this section by introucing the notion of an SD systems using the concept of an SD operator. k Fig. 4. Topology of the system on an arbitrary connecte graph. Coupling between two agents is shown by an unirecte ege between them. Definition 8: The system (2)-(3) is calle spatially ecaying (SD) if the state-space operators A, B, C, D are SD with respect to the one-parameter family of coupling characteristic functions C. VI. EAMPLES OF SPATIALLY DECAYING OPERATORS The following class of operators which are use extensively in cooperative an istribute control are interesting special classes of SD operators. A. Spatially Truncate Operators These are operators with finite range couplings. Given the coupling range T >, the following class of linear operators are SD with respect to every coupling characteristic functions χ α [Q] ki = Q ki if is(k, i) T if is(k, i) > T T i (18) where Q ki C n n. For this case, some common choices for the istance function are Eucliean istance an geoesic or minimum hop count istance (i.e., hop count on the shortest = path). For such operators an every given noe k G, we have that [Q] ki χ α (is(k, i)) [Q] ki χ α (T ) <. (19) i k The relation is the neighborhoo relation efine as i k if an only if is(k, i) T. Inequality (19) shows that Q is SD with respect to every C an the ecay margin is any real number b >. Examples of such operators arise in motion coorination of autonomous agents such as the Laplacian operator [39]. Suppose that G is a connecte istance epenent proximity graph with the set of noes G an the set of eges E. Furthermore, suppose that eges are weighte with a given weighting function w : E C n n, for example, w ki is the weight on ege connecting noes k an i. Let f : G C be a function mapping vertices to complex numbers. Then the
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 8 = iscrete Laplacian operator L w is efine by (L w f)(k) w ik (f(i) f(k)) (2) i k in which the neighborhoo relation is efine as above. By selecting the =8< minimum hop count istance function, the matrix representation of Laplacian operator will be : k w kk if k = i [L w ] ki w ik if is(k, i) = 1 (21) otherwise in which k is the egree of noe k. Such an operator is obviously SD. B. Exponentially Decaying Operators Consier the one- parameter family of coupling characteristic functions C E efine by χ ζ (s) = (1 + ζ) s (22) where ζ R +. Operator Q is sai to be exponentially SD if conition (17) hols with respect to C E efine by (22) for all ζ [, b). Here b > is the ecay margin. The first obvious example of such operators is operators with finite range couplings such as the one efine by (18) for some T >. For every noe k G, we have [Q] ki χ ζ (is(k, i)) [Q] ki (1 + ζ) T. (23) i k The right han sie of (23) is a polynomial in terms of ζ with maximum egree T. Since every polynomial is finite on any interval [, b) with b >, the right han sie of (23) is boune, an that Q is exponentially SD with ecay margin b = +. Another important example of exponentially SD operators are the class of translation invariant operators. The result of theorem 3 along with the immeiate application of lemma 1 shows that a translation invariant operator in L(l 2 ) is exponentially SD. In this case, since the interconnection topology is assume to be a lattice, the suitable choice of a istance function is is(k, i) = k i. Applying the results of theorem 3, for every k G, by selecting any ζ [, e β 1) where < β < ln(1 + r), it follows that e β k i (1 + ζ) k i i Z [Q] ki (1 + ζ) is(k,i) α i Z α e γ + 1 e γ 1 < where γ = β ln(1 + ζ) is a positive number. The ecay margin of Q is equal to r, the istance of the nearest pole of ˆQ(z) to the unit circle. C. Algebraically Decaying Operators Consier the parameterize family of characteristic functions C A efine as χ ν (s) = (1 + λs) ν (24) in which λ > an ν R +. Operator Q is sai to be algebraically SD if conition (17) hols with respect to C A efine by (24) for all ν [, b) where b > is the ecay margin. Such functions are often use as pair-wise potentials among agents in flocking an cooperative control problems [4]. The intuition behin such a coupling is that the interaction between two subsystems in a networke system can be moelle as a non-increasing function of their Eucliean istance. Another example of such coupling functions arises in wireless networks. The coupling between noes, which is consiere as the power of the communication signal between agents, ecays with the inverse fourth power law [41], i.e., 1 is(k, i) 4. In such applications, the Laplacan operator L w (or the corresponing ajacency operator) can be efine with the following weighting function w ki = for some ν an λ >. 1 χ ν (is(k, i)) VII. PROPERTIES OF SD OPERATORS Theorem 4: Given a one-parameter family of coupling characteristic functions C, the set of all linear operators that are SD with respect to C with ecay margin at least b > forms a Banach Algebra. Proof: See appenix I-A for a proof. For a comprehensive iscussion on Banach algebras we refer the reaer to any Functional Analysis textbook, for example [34]. This Banach algebra is enote by in which the operator norm Q b = S b (C ) = {Q : Q b < } sup sup α [,b) [Q] ki χ α (is(k, i)) satisfies the usual conitions, i.e., for all Q, P S b (C ) an c C, (1) Q b an Q b = iff Q =, (2) c Q b = c Q b, (3) Q + P b Q b + P b. Furthermore, it is submultiplicative, (4) QP b Q b P b. Accoring to the efinition, the operator space S b (C ) equippe with norm. b is a Banach algebra. The above theorem is a key ingreient in proving that optimal controllers of SD systems are SD. However, we first nee the following result on closure uner limit property of S b (C ). This result is require for stuying the properties of the unique solution of Lyapunov equation an ARE through the stuy of associate ifferential equations. In the next section, using the closure uner limit property of SD operators, it is
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 9 shown that the solution of ifferential Lyapunov an Riccati equations converge to an SD operator. Theorem 5: Let C be a one-parameter family of coupling characteristic functions with ecay margin b >. Consier the one-parameter family of operator-value functions P(t) : R + L(l p ) with the following properties: (1) lim t P(t) P lp l p =, (2) P(t) S b (C ) for all t. Then lim t P(t) P b =. Furthermore, P S b(c ). Proof: See appenix I-B for a proof. To summarize, we have shown that operator space S b (C ) is close uner aition, multiplication, an limit properties. Furthermore, it turns out that if an SD operator has an algebraic inverse on L(l ), the inverse operator Q 1 is also SD [?]. Remark 3: Using the above results, it is straightforwar to check that the serial an parallel composition of two SD systems are SD. Furthermore, if the feeback interconnection of two SD systems t ψ i(t) = (A i ψ i ) (t) + (B i u i ) (t) y i (t) = (C i ψ i ) (t) + (D i u i ) (t) for i = 1, 2, is well-pose [42] an if operator I D 2 D 1 has an algebraic inverse in L(l ), then the feeback interconnection of these two systems is also SD. VIII. STRUCTURE OF QUADRATICALLY OPTIMAL CONTROLLERS As iscusse in section III, our aim is not to solve the Lyapunov equation (5) an ARE (7) explicitly but to stuy the spatial structure of the solution of these algebraic equations by means of tools evelope in the previous sections. In the following, it is shown that the solution of equations (5) an (7) have an inherent spatial locality an the characteristics of the coupling function will etermine the egree of localization. A. Lyapunov Equations In the following, it is shown that for SD systems that are stable an are escribe by (4), solution of the Lyapunov equation P is also SD. The proof sketch is as follows: first, we consier the corresponing Lyapunov ifferential equation an show that the solution is SD for every time instant. Then using the closure properties of the Banach Algebra S b (C ), it is conclue that P is SD with respect to C. Theorem 6: Assume that operators A, Q S b (C ) an Q is positive efinite. If A is the infinitesimal generator of an exponentially stable C -semigroup T (t) on l 2, then the unique positive efinite solution of Lyapunov equation Aφ, Pφ + Pφ, Aφ + φ, Qφ = (25) for all φ l 2, satisfies P S b (C ). Proof: First, we will prove that the C -semigroup T (t) with infinitesimal generator A is SD with respect to C. The following is a stanar result from [35]: T (t)φ = A T (t)φ (26) t with T () = I an for all φ l 2 an t >. Therefore, for all k, i G we have t [T (t)] ki = [A] kj [T (t)] ji. (27) j G For a ifferentiable matrix (t) C n n for t, we have the following inequality (t + τ) (t) (t) = lim t τ τ + τ) (t) lim τ (t τ (t + τ) (t) lim τ τ t (t). (28) Assume that T (t) is a solution of (26), using inequality (28), we have t sup [T (t)] ki χ α(is(k, i)) k G sup sup t [T (t)] kiχ α (is(k, i)) [A] kj [T (t)] jiχ α (is(k, i)). j G Using the fact that. b is submultiplicative, from the above inequality we can conclue that t T (t) b an it follows that A b T (t) b T (t) b T () b e A b t (29) for all t. Note that T () b = 1. Because operator A is SD with respect to C with ecay margin b, accoring to (29) the family of one-parameter operators satisfy T (t) S b (C ) for all t. Now consier the ifferential form of the Lyapunov function φ, P(t)φ = Aφ, Pφ + Pφ, Aφ + φ, Qφ t with P() = Q for all φ l 2. This equation has a solution of the following form (cf. [35]) P(t) φ =Zt T (σ) Q T (σ) φ σ (3) for all φ l 2. Therefore, for every k, i G, we have [P(t)] ki =Zt [T (σ) Q T (σ)] ki σ. (31)
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 1 1Š Accoring to inequality (29) an equation (31) an using the fact that. b is submultiplicative, we get e 2 A P(t) b Q b t b 2 A. (32) b Therefore, P(t) S b (C ) for all t. On the other han, solution of the ifferential Lyapunov equation (3) converges to the unique solution of (25), i.e., lim P(t) P l 2 l 2 =. t Accoring to Theorem 5, it follows that lim P(t) = P t uniformly in S b (C ). Therefore, P S b (C ). B. Algebraic Riccati Equation As iscusse in Section III, the key ingreient in solution of linear optimal control problems in finite an infiniteimensional case is fining the solution to the corresponing Riccati equation. In the following, we will show that the solution of the Riccati equation for SD systems as well as the kernel of optimal feeback associate to an SD system is itself an SD operator, K = R 1 B P. (33) Without loss of generality, we will assume that R = I. Otherwise, by only assuming that R has an algebraic inverse on L(l ), it can be shown that R 1 is SD [?]. Accoring to the closure uner multiplication property of SD operators, if P is SD, then K will be SD. The proof of the following theorem, is more or less similar to the proof of the theorem 6. Theorem 7: Let A, B, Q S b (C ) an Q. Moreover, assume that conitions of theorem 2 hol. Then the unique positive efinite solution of the following ARE ϕ, PA φ + PA ϕ, φ + ϕ, Q φ B P ϕ, B P φ = for all ϕ, φ l 2, satisfies P S b (C ). Proof: Consier the following Differential Riccati Equation (DRE) t ϕ, P(t)φ = ϕ, P(t)Aφ + P(t)Aϕ, φ + ϕ, Qφ B P(t)ϕ, B P(t)φ with P() =. We enote the unique solution of this ifferential Riccati equation in the class of strongly continuous, self-ajoint operators in L(l 2 ) by the one-parameter family of operator-value function P(t) for t. The nonnegative operator P, the unique solution of ARE, is the strong limit of P(t) on l 2 as t (see theorem 6.2.4 of [35]). Therefore, we have that lim P(t) P l 2 l 2 =. (34) t From the ifferential Riccati equation, it follows that t [P(t)] ki = [A P(t) + P(t)A P(t)BB P(t) + Q] ki for all k, i G. Using inequality (28), we have sup t P(t) b = sup [P(t)] ki χ α (is(k, i)) t [P(t)] ki χ α (is(k, i)) A P(t) + P(t)A P(t)BB P(t) + Q b. For simplicity in notations, enote π(t) = P(t) b. Using the triangle inequality an the fact that norm. b is submultiplicative, we have the following ifferential inequality π(t) 2 A b π(t) + ( B b) 2 π(t) 2 + Q b (35) with initial conition π() = an constraint π(t) for all t. All coefficients A b, B b, Q b in the right han sie of the inequality (35) are finite numbers. If π(t) for t is a solution of the ifferential inequality (35), then it is also a solution of the following ifferential inequality π(t) λ (π(t) + 1) 2 (36) with initial conition π() =, in which λ = max( A b, ( B b) 2, Q b). In other wors, the set of feasible solutions of (35) is a subset of solutions of (36). From (36), we have t 1 π(t) + 1 λ which has the following set of solutions 1 π(t) + 1 e λt π() + 1. Using the fact that π(t) for all t an π() =, it follows that π(t) e λt 1. The above inequality is feasible, i.e., there exists at least one sequence of solutions satisfying π(t) for all t. The above inequality also proves that π(t) < for all t. Therefore, we have that P(t) S b (C ) for all t. Accoring to theorem 5, we can use this result an (34) to conclue that P S b (C ). This completes the proof. I. SIMULATION We consier a large network of N linear subsystems couple on a one-imensional chain which can be escribe by ψ(t) = (Aψ) (t) + (Bu) (t). t The coupling characteristic function is χ an the system operators are given by [A] ki = 1 χ(is(k, i))
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 11 3 25 2 15 1 5 5 36 32 12 8 39 43 9 15 31 21 26 2 29 3 24 27 25 23 4 6 16 46 14 1 33 4 11 48 28 19 7 22 3 45 47 38 37 44 35 41 34 13 2 17 18 1 5 Norm of LQR Feeback Gain 3 2 1 3 2 1 3 2 1 3 2 1 Subsystem No. 1 5 1 15 2 25 3 Subsystem No. 4 2 4 6 8 1 12 14 16 18 2 Subsystem No. 12 5 1 15 2 25 3 35 Subsystem No. 15 42 49 5 1 15 2 25 3 5 1 15 2 25 Distance Fig. 5. N=5 noes are ranomly an uniformly istribute in a region of area 3 3 (units) 2. Each noe is a linear subsystem which is couple to other subsystems through their ynamic an a central cost function by a given coupling characteristic function. [K] Fig. 6. Norm of LQR feeback gain [K] ki (bar) an kk χ ν (is(k,i)) when λ =.1 an ν = 4 (ashe) for subsystems k = 1, 4, 12, 15, respectively, from top to bottom. an B = I. The istance function is Eucliean. We will stuy the LQR problem iscusse in Section III with weighting operators R = I an Q being the corresponing graph Laplacian given by [Q] ki = N 1 if k = i 1 if k i. The corresponing ARE is given by A P + PA P 2 + Q =. (37) Then the LQR optimal feeback is given by K = P. (38) In the following simulations, it is assume that N = 5 noes are ranomly an uniformly istribute in a region of area 3 3 (units) 2. Each noe is assume to be a linear system which is couple through its ynamic an the LQR cost functional to other subsystems. In the sequel, three ifferent scenarios are consiere for the coupling characteristic function. A. Algebraical Decay The first simulation is one base on the coupling characteristic functions of algebraical type given by (24) with parameters λ =.1 an ν = 4. In Figure 6, the norm of the LQR feeback gains (38) corresponing to agents k = 1, 4, 12, 15 (their locations are marke by bol stars in Figure 5) is epicte versus the istance of other subsystems to subsystem k. As seen from these simulations, for every subsystem k the norm of the optimal feeback kernel [K] ki is upper [K] boune by function kk χ ν (is(k,i)). Therefore, the spatial ecay rate of the optimal controller can be etermine priory only using the information of the coupling characteristic function χ ν (is(k, i)). As seen in Figure 6, for each subsystem k, the corresponing optimal controller is effectively couple only to those subsystems (with inex i s) for which is(k, i) 1 (units). This suggests the possibility of formulating the optimal control problem in a istribute fashion, rather than solving a centralize high-imension algebraic equation such as (37) (see [27]). B. Exponential Decay In the next simulation, the exponential coupling characteristic functions given by (22) are investigate. The simulation parameter is selecte as ζ = e 1. Figure 7 shows the norm of LQR feeback gains (38) corresponing to agents k = 1, 4, 12, 15 versus the istance of other subsystems to subsystem k. For subsystem number k, it can be seen that norm of the optimal feeback gains are upper boune by function [K] kk χ ζ (is(k,i)) which implies that [K] ki ecays exponentially as istance increases. Accoring to simulations, the coupling strength between subsystems are negligible for those which are beyon istance of approximately 5 (units) from each other. Therefore, spatial truncation can be performe aroun each subsystem beyon raius of 5 (units). C. Nearest Neighbor Coupling In the last simulation, the nearest neighbor coupling case with T = 5 is stuie, as iscusse in Section VI. Figure 8 represents the norm of LQR feeback gains (38) corresponing to agents k = 1, 4, 12, 15 versus the istance of other subsystems to subsystem k. In this case, the coupling strength is negligible between ynamically uncouple subsystems, i.e., subsystems with a istance greater than T = 5 (units). In fact, by selecting a nearest neighbor coupling policy, we impose a specific architecture on the network. Simulation results affirm that the optimal controller inherits the same architecture as the unerlying system. Naturally the unerlying solution is SD both with respect to algebraic an exponential coupling characteristic function D. Spatial Truncation Let K T be the spatially truncate operator efine as follows [K T ] ki = [K] ki if is(k, i) T if is(k, i) > T.
SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 12 3 4 2 1 Subsystem No. 1 2 Subsystem No. 1 Norm of LQR Feeback Gain 3 2 1 3 2 1 5 1 15 2 25 3 Subsystem No. 4 2 4 6 8 1 12 14 16 18 2 Subsystem No. 12 Norm of LQR Feeback Gain 4 2 4 2 5 1 15 2 25 3 Subsystem No. 4 2 4 6 8 1 12 14 16 18 2 Subsystem No. 12 3 2 1 5 1 15 2 25 3 35 Subsystem No. 15 4 2 5 1 15 2 25 3 35 Subsystem No. 15 5 1 15 2 25 Distance 5 1 15 2 25 Distance [K] Fig. 7. Norm of LQR feeback gain [K] ki (bar) an kk χ ζ (is(k,i)) when ζ = e 1 (ashe) for subsystems k = 1, 4, 12, 15, respectively, from top to bottom. Fig. 8. Norm of LQR feeback gain [K] ki (bar) an [K] ki pulse function with length T = 5 (ashe) for subsystems k = 1, 4, 12, 15, respectively, from top to bottom. Applying the small-gain stability argument, it is straightforwar to verify that in the above simulations the truncate feeback K T is stabilizing if T T s in which: T s = 4.5 for algebraical ecay. T s = for exponential ecay. T s = 5 for nearest neighbor coupling. Figure 9 illustrates the performance loss percentage efine as λ max (P P T ) 1 λ max (P) versus ifferent values of T T s for ifferent coupling characteristic functions. As seen from Figure 9, the larger values of truncation length T ensue better close-loop performance. For example, a 1% performance loss trae off results in to the following stabilizing truncation lengths: T = 13.2 (units) for algebraical ecay. T = 2.1 (units) for exponential ecay. T = 5.2 (units) for nearest neighbor coupling.. CONCLUSIONS In this paper we stuie the spatial structure of infinite horizon optimal controllers for spatially istribute systems. By introucing the notion of SD operators we extene the notion of analytic continuity to operators that are not spatially invariant. Furthermore, we prove that SD operators form a Banach algebra. We use this to prove that solutions of Lyapunov an Riccati equations for SD systems are themselves SD. This result was utilize to show that the kernel of optimal LQ feeback is also SD. Although these results were proven for LQ problems, they can be easily extene to general H 2, an H optimal control problems as the key enabling property is the spatial ecay of solution of the corresponing Riccati equations. One major implication of these results is that the optimal control problem for spatially ecaying systems lens itself to istribute solutions without too much loss in performance as even the centralize solutions are inherently localize. These results have been extene to the case of Performance Loss Percentange % 5 4 3 2 1 5 1 15 2 25 3 35 5 4 3 2 1 5 1 15 2 25 3 35 2 15 1 5 5 1 15 2 25 3 35 Length of Truncation Algebraical Decay Exponential Decay Nearest Neighbor Coupling Fig. 9. Performance Loss percentage of LQR controller after spatial truncation for ifferent types of couplings: (i) algebraical ecay (ii) exponential ecay (iii) nearest neighbor coupling. constraine finite horizon optimal control problems by blening the ieas evelope here with Multi Parametric Quaratic Programing [32], [33]. One important future research irection is to further stuy the case of SD operators with finite support (e.g., systems with nearest neighbor coupling). It woul be interesting to fin out uner what extra conitions the solutions are themselves finite support, as oppose to just being spatially ecaying. This woul provie an interesting connection between these results an those of [3]. A. Banach Algebra S b (C ) I. APPENDI Proof: Properties (1) an (2) are immeiate from the efinition. To prove (3), we use the following chain of in-