UNIVERSITY of CALIFORNIA Santa Barbara. Communication scheduling methods for estimation over networks

Transcription

1 UNIVERSITY of CALIFORNIA Santa Barbara Communication scheduling methods for estimation over networks A Dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Electrical and Computer Engineering by Yonggang Xu Committee in charge: Professor João P. Hespanha, Chair Professor Francesco Bullo Professor Roy S. Smith Professor Andrew R. Teel March 2006

2 The dissertation of Yonggang Xu is approved. Professor Francesco Bullo Professor Roy S. Smith Professor Andrew R. Teel Professor João P. Hespanha, Committee Chair November 2005

3 Communication scheduling methods for estimation over networks Copyright c 2006 by Yonggang Xu iii

4 To the memory of my father Shuiming Xu ( ) iv

5 Acknowledgements For the completion of this work I am grateful to my adviser Professor João Pedro Hespanha for his support and guidance. João has much for me to learn from, professionalism and dedication, among many others. The knowledge and skills I learned from him will be my great assets for all the years to come. Never would I forget of his warmth the first time I went to his office at the end of It has been a great experience to be a member of Center for Control Engineering and Computation in UC Santa Barbara. From the professors, I learned not only knowledge, but also the willingness to share and collaborate. I am grateful to Professor Peter Kokotović, who showed me trails in Santa Barbara mountains and helped me making important decisions. I also want to thank Professor Brad Paden for having me here in UCSB. Professors on my committee, Francesco Bullo, Roy Smith, and Andrew Teel, spent much of their Thanksgiving Holiday to review the dissertation draft. Their suggestions and critiques are essential and greatly appreciated. All my colleagues and friends made my study and life so memorable in UCSB. On whiteboards, we studied real analysis and jump systems. At home we enjoyed international potluck dinners and birthday parties. On the local mountains and Grand Canyon trails were our footprints. I owe the most to my family, my grandma, my late father, my mother, and my sister Yan. My parents worked hard and made every attempt for their children; they earned little yet managed to bring the family a vision, for which my dad made the ultimate sacrifice... Xiaoling supported me with love during various phases of the dissertation. v

6 Curriculum Vitæ Yonggang Xu Yonggang Xu was born in Xiaoshan County, Zhejiang Province, China, in EDUCATION 1998 B.S. Vehicle Engineering. Tsinghua University, Beijing, China M.S. Vehicle Engineering. Tsinghua University, Beijing, China M.S. Electrical Engineering. University of California, Santa Barbara PROFESSIONAL EMPLOYMENT Graduate Researcher, Tsinghua University, Beijing Teaching Assistant, University of California, Santa Barbara Research Assistant, University of California, Santa Barbara. SELECTED PUBLICATIONS Y. Xu and J.P. Hespanha. Optimal communication logics for networked control systems. In Proceedings of the 43rd IEEE Conference on Decision and Control. pp , Dec Y. Xu and J.P. Hespanha. Design and analysis of communication logics in networked control systems. Book chapter in Current Trends in Nonlinear Systems and Control. Birkhauser, vi

7 Y. Xu and J.P. Hespanha. Estimation under uncontrolled and controlled communications in networked control systems. In Proceedings of the 44rd IEEE Conference on Decision and Control. pp , Dec J.P. Hespanha, P. Naghshtabrizi, and Y. Xu. A Survey of recent results in networked control systems. To appear in the Proceedings of IEEE. vii

8 Abstract Communication scheduling methods for estimation over networks by Yonggang Xu This work focuses on data communication scheduling methods for estimation over networks. A communication scheduler is a mechanism that determines when a data packet should be sent to the network, given the information pattern available. For stochastic communication schedulers, decisions to send data are based on outcomes of random processes. Two issues are addressed in the dissertation: communication scheduler analysis and communication scheduler optimization. We propose various types of stochastic communication schedulers based on counting processes. We model the dynamics as jump-diffusion processes and use stochastic analysis methods based on infinitesimal generators. The networks we consider include erasure networks and delayed networks. We construct optimal stochastic communication schedulers that minimize network usage and maximize estimation performance. We formulate a long-term average cost problem with an unbounded per-stage cost function on a Borel state space, which is solved by dynamic programming techniques. The results shed light on how to schedule data communication for networked control systems (NCSs). Another contribution is on data rate limitations. We derive conditions for the existence of stable estimators under both the bit rate and the packet rate viii

9 limitations. The conditions show a trade-off relation between the bit rate and the packet rate. We observe that if the packet size is large, then increasing it (and therefore increasing the bit rate) helps little, and the packet rate becomes the dominant factor. This observation encourages us to think in terms of packets instead of bits, as we do in most parts of the dissertation. ix

10 Contents Acknowledgments Curriculum Vitæ Abstract List of Figures v vi viii xiv 1 Introduction Motivation Estimation over network problems Communication schedulers Literature review Sampling and quantization Minimum bit rate Minimum packet rate Dissertation research Summary Rate limitations Rate limitations for LTI systems Problem setup One-dimensional noiseless LTI systems x

11 2.1.3 One-dimensional LTI systems with noise Multi-dimensional LTI systems Special cases Large packet sizes Lossless networks Erasure networks An example for rate limitations From bits to packets Bit rate vs. packet rate Packet rate and communication performance Summary Jump-diffusion processes Counting process and its intensity function Solutions to jump-diffusion equations Infinitesimal generators Communication schedulers Definitions and Motivation Definitions Why stochastic communication schedulers? Remote estimation with local state measurement A remote estimation scheme Error dynamics Uncontrolled communications Controlled communications τ-delayed networks Remote estimation with local output measurement A communication-estimation scheme Error dynamics Uncontrolled communications xi

12 4.3.4 Controlled communications Controlled communications: saturated intensities Periodic communication over lossy networks Discussions Deterministic communication schedulers Simulations Rate-variance curves Remote estimation with local output measurement Summary Optimal communication schedulers Motivation Stochastic communication scheduling methods Cost criterion Problem statement Information patterns Problem formulation Communication-estimation structure Structured reformulation Error dynamics Optimal communication problem Lossless networks Main results Value iteration Erasure networks Main results Upper bound Lower bound Numerical examples Summary xii

13 6 Summary and future work 109 Bibliography 112 A Concepts in stochastic processes 120 B Technical proofs 123 B.1 Proof of theorems in Chapter B.2 Proof of theorems in Chapter B.2.1 Proof of Lemma B.2.2 Proof of Lemma xiii

14 List of Figures 1.1 General NCS architecture Two remote estimation schemes A remote estimation scheme with a communication scheduler Estimation over networks Graphical representation of Theorem Packet rate vs. bit rate Remote estimation with local state measurement A remote estimation scheme: local Kalman filter state is sent A remote estimate scheme: measurements are sent The detailed structure of the estimation scheme Packet rate vs. estimation error variance Estimation error variances under different Poisson rates Variances (left) and packet rates (right) with jump intensity ẽ P ẽ Information patterns A structured solution to Problem Numerical solution for a lossless network Numerical solution for a network that drops packets xiv

15 Chapter 1 Introduction Networked Control Systems (NCSs) are spatially distributed systems in which communication between sensors, actuators, and controllers is supported by a shared communication network (see Figure 1.1). PSfrag replacements Node 1 Node M Sensors Plant Actuators Sensors Plant Actuators Enc Dec Enc Dec Network Controller Controller Figure 1.1. General NCS architecture. In NCSs feedback is implemented by information flow over data networks. The research involves both the classic information theory and control theory. Murray et al. [32] identify control over networks as one of the key future directions for control. Readers are referred to [2, 17] for a general review. NCSs have found numerous applications in a broad range of areas such as 1

16 mobile sensor networks [37], remote surgery [31], haptics collaboration over the Internet [16, 20, 48], and automated highway systems and unmanned aerial vehicles [45, 46]. This dissertation focuses on data communication scheduling methods for NCSs. A communication scheduler 1 is a mechanism that determines whether or not a node sends data to the network given information available. By the term node, we mean locally interconnected components, e.g., a plant, its sensors/actuators, and its encoder/decoder (see Figure 1.1). The mechanism can be either deterministic or stochastic. For a deterministic communication scheduler, the decision to send data is a deterministic function of the given information set. For a stochastic scheduler, the decision is based on outcomes of a certain random process. We will propose, analyze, and optimize communication schedulers. Section 1.1 is on the motivation of the dissertation, from the viewpoint of network applications. In Section 1.2, we survey the literature on NCSs with emphasis on estimation over networks and minimum packet rate. In Section 1.3, we will give an overview of the main dissertation work. Finally, Section 1.4 outlines the chapters to come. 1 In the previous papers, we used the terminology communication logic. The logic schedules when or whether to send data packets. In this perspective, the term scheduler may be more appropriate, which is adopted throughout the dissertation. 2

17 1.1 Motivation The dissertation is on communication schedulers for estimation over networks. We give motivation behind this work Estimation over network problems The applications of estimation over networks include remote sensing, space exploration, and sensor networks, among others. It is also a crucial component of certainty equivalence NCS controllers, which construct control signals based on state estimates of a remote systems. In several NCS scenarios, certainty equivalence controllers are not optimal, but they are still of a great practical interest due to the difficulties in designing optimal controllers. In the dissertation, we only consider estimation over networks. However, we can extend the idea to control problems. In [60], Xu and Hespanha consider stabilizing certainty equivalence controllers for spatially distributed processes whose dynamics is decoupled but whose control objectives are not, e.g., a group of autonomous aircraft flying in a geometric formation far enough from each other to have decoupled dynamics. In [51], Smith and Hadaegh consider the control of spacecraft, in which parallel estimators are used to construct local controls and estimators are updated via communication among the formation entities. They note that each craft having an estimate of the entire formation makes it possible to find an optimal control for the whole formation and for each craft to implement higher level functions such as collision avoidance. 3

18 1.1.2 Communication schedulers The main reason to study communication schedulers is that data is transmitted packet by packet in most communication systems. We emphasize that a node makes communication decisions actively. We aim to reduce the communication load by properly choosing communication schedulers. In general, reducing packet rate (network load) improves the real-time network performance by reducing the packet dropout probability, shortening communication delays, and saving radio power, among others. Communication schedulers can either be applied to existing communication protocols to better serve NCSs or be implemented in new protocols for NCSs applications. For the input-output stability analysis of NCSs for a broad class of medium access control (MACs) methods, the readers are referred to [36]. However, our emphasis is to explore the application potential of communication schedulers. In carrier sense multiple access (CSMA) protocols, upon detecting that a collision happens, a sending node may choose to send data in a persistent or non-persistent fashion [54]. One algorithm is called the p-persistent CSMA, in which the node sends a packet with probability p, 0 p 1. An important issue is to choose the value of u, i.e., the probability of data re-sending immediately after a collision. Some research efforts are on how to choose the parameter p to improve network performance. It is shown via simulation in [8] that the predictive p-csma achieves a smaller collision rate, a lower mean packet delay, and a higher throughput under most traffic conditions. We extend this idea in our framework, which provides a mechanism to choose 4

19 data sending probability u according to how stringent data communications are. Network usage is saved to nodes with highest priorities, while collision probability is reduced in the network overall. As we shall see in the dissertation, the parameter u pertains to the intensity function of a stochastic communication scheduler (Chapters 4) or the optimal communication policy (Chapter 5). 1.2 Literature review There is an extensive list of NCSs research topics: sampling and quantization effects, encoder and decoder, minimum bit rate, minimum packet rate, network dynamics analysis, network protocol design, and controller design, among others. We will examine a few Sampling and quantization Sampling schemes can either be time-driven or event-driven. Åström and Bernhardsson [53] compare the merits of the Riemann sampling (time-driven) and the Lebesgue sampling (event-driven) for one-dimensional systems. Yook et al. [63] propose that a node should broadcast the true value of the local plant state when it differs from the estimate known to the remote nodes by more than a given threshold. They show that this scheme results in a system that is bounded-input bounded-output stable. The relation between the threshold level and the message exchange rate is investigated through simulations. A signal has to be quantized before being encoded and sent to a digital channel. A quantizer is a device that converts real numbers (an analog signal) into 5

20 a finite set of integers (a digital signal). Mathematically, a quantizer is a piecewise constant function, mapping a quantization region to a quantization point. Usually, a quantization region is a pre-specified rectangular shape. More efficient quantization schemes (for controls) have been developed over the years. Both [13] and [21] advocate logarithmic-based quantization methods. Roughly speaking, quantization region becomes larger logarithmically as the distance from the origin grows. Quantization regions are allowed to evolve with time to capture the system dynamics [18]. Bullo and Liberzon [7] focus on designing the least destabilizing quantizer subject to a given information constraint. They show that quantizer design can be reduced to a version of the so-called multicenter problem from locational optimization, for which they develop iterative algorithms. We use event-driven sampling methods for large part of the dissertation. And therefore it is related to [53] and [63]. Quantization is not the emphasis of the dissertation, as we assume that a data packet holds a real number with a precision sufficient for control or estimation purposes. However, we do allow bounded quantization errors in our analysis Minimum bit rate Any communication network can only carry a finite amount of information per unit time. Inspired by Shannon s results on the maximum bit rate that a finitebandwidth channel can reliably carry, a significant research effort has been devoted to the problem of determining the minimum bit rate to stabilize a system through feedback. The problem of determining the minimum information flow 6

21 needed for stabilization has been solved exactly for linear plants [59, 18, 34, 35, 57, 43] and preliminary results have been obtained for nonlinear plants [33, 28]. Other issues, such as robustness, have also been addressed [40, 27, 29, 55]. We will come back to this topic in Chapter 2. But we intend to depart from bit rate to the so-called packet rate Minimum packet rate In digital communication networks, data is transmitted in atomic units called packets 2. In parallel to bit rate, packet rate refers to the average number of data packets per unit time. We review estimation over lossy networks problems to demonstrate the effects of the packet rate. We consider erasure networks, in which a data packet can carry a real number without distortion, but that some packets may be lost. Two scenarios for state estimation over networks are investigated. In the one depicted in Figure 1.2(a), every raw sensor measurement y t is sent to the remote estimator, but may not arrive there if there is a packet dropout [30, 50, 49]. Alternatively, in the scenario shown in Figure 1.2(b), the raw sensor measurements are processed locally and then sent to the remote estimator via the same network [60, 63, 62]. We restrict our attention to linear time-invariant (LTI) plants with Gaussian measurement noise and disturbance: x t+1 = Ax t + w t, y t = Cx t + v t, t N, x t, w t R n, y t, v t R l, (1.1) where the initial state x 0 is zero-mean Gaussian with covariance matrix Σ, and 2 Here the term packet is generic. Depending on different networks, it may take the form of (variable-size) packet or (fixed-size) cell, among others [58]. 7

22 PSfrag replacements Estimator Plant y t Linear plant Network Estimator Remote Plant y t Loc. est Smart sensor x t Network Estimator Remote (a) (b) Figure 1.2. Scenarios for the state estimation over a network. In (a) the raw sensor measurements are sent to a remote estimator, whereas in (b) the measurements are processed locally before transmission. the zero-mean Gaussian white noises w t and v t are mutually independent with covariance matrices Σ w 0 and Σ v > 0, respectively. We assume that (C, A) is detectable and (A, Σ w ) is stabilizable. We discuss the minimum packet rates required to construct estimators for configurations in Figure 1.2(a) as well as Figure 1.2(b). Minimum packet rate without local calculation First we consider the architecture in Figure 1.2(a), in which the measurements, y t, are sent. The optimal estimate of x t (t N) can be computed recursively using the following time-varying Kalman filter (TVKF) [23]: ˆx 0 1 = 0, ˆx t t = ˆx t t 1 + ν t L t (y t Cˆx t t 1 ), t N, (1.2) ˆx t+1 t = Aˆx t t, 8

23 where the matrix gain L t is calculated recursively as follows P 0 = Σ, L t = ν t P t C (CP t C + Σ v ) 1, t N, P t+1 = AP t A + Σ w AL t (CP t C + Σ v )L ta, where P t ( t N) is an estimation error covariance matrix, i.e., P t = E[(x t ˆx t t 1 )(x t ˆx t t 1 ) ]. Sinopoli et al. [49] study the performance of this Kalman filter when ν t is a Bernoulli process with probability of dropout (ν t = 0) equal to p [0, 1). They show the existence of a critical value p c for the dropout rate p, above which the estimation error covariance matrix becomes unbounded. The critical value p c satisfies p p c p, where the upper bound, p, is given by p = 1 (max{ eig(a) }) 2, (1.3) and the lower bound, p, is given by the solution to a linear matrix inequality. In special cases (e.g., the matrix C is invertible) the upper bound in (1.3) is tight in the sense that p c = p. Matveev and Savkin [30] consider multiple sensors, each independently sending its measurements to the estimator with some delay. This corresponds to the following plant model, x t+1 = Ax t + w t, y s,t = C s x t + v s,t, s {1,, N}, t N, (1.4) where y s,t denotes the measurement collected by sensor s at time t. Assuming that the measurement y s,t suffers a random delay of τ s (t), the optimal state 9

24 estimate at time t is given by ˆx t = E [ x t ys,l, l t τ s (l) s.t. θ s,l = 1 ], where θ s,l = 1 if y s,l reaches its destination, and θ s,l = 0 otherwise. They derive a recursive Kalman filter and provide conditions under which the estimation error process is almost surely stable [30]. These conditions are given in terms of the observability of x t for specific realizations of the process θ s,t. Minimum packet rate for estimation with local computation The encoder-estimator scheme in Figure 1.2(b) is motivated by the growing number of smart sensors with embedded processing units that are capable of local computation. In this context, [62] investigates the benefits of pre-processing measurements before transmission to the network. For the LTI plant (1.1), the smart sensor computes locally an optimal state estimate, x t t = E[x t t y s, s t], using a standard Kalman filter. The local node then transmits the local estimate x t t (instead of the raw measurement y t ), which is used by the remote estimator to compute the optimal estimate ˆx t s of x t (t N) given all the data { x r r : ν r = 1, r s} successfully received up to time s t, ˆx t s = E [ x t x r r, r s s.t. ν r = 1 ]. The main advantage of using local computation is that each message x t t that successfully reaches the remote estimator encodes all the relevant information that can be extracted from all raw measurements collected up to time t. In general, this permits stability of the error process e t for larger dropout probabilities than those permitted by the architecture in Figure 1.2(a), in which raw measurements are sent over the network. We will discuss further in Chapter 4. 10

25 1.3 Dissertation research We characterize the main dissertation work as communication scheduler analysis and communication scheduler optimization. We also extend results on the minimum data rate problem, in which we consider both the bit rate and the packet rate. Now we take snapshots of the main results with informal explanations in the first part of this section. In the second part, we state the motivation from the viewpoint of control networks. We mainly discuss estimation (of LTI systems) over networks problems. The estimation scheme consists of a local node, a remote estimator, and a network, as shown in Figure 1.3. The local node has a local estimator with the state x(t) and a communication scheduler. The communication scheduler decides whether or PSfrag replacements not a data packet should be sent to the network given information available. We focus on stochastic communication schedulers, in which decisions to send data are based on outcomes of a certain random process. Plant x(t) y(t) Encoder c(t) x(t t) Network Estimator ˆx(t) N(t) Loc. Est. x(t) Local node Comm. scheduler Remote estimator Acknowledgment Figure 1.3. A remote estimation scheme with a communication scheduler Communication scheduler analysis In the continuous-time domain, a communication scheduler is modeled as a counting process (Chapter 3) with a certain intensity function. The intensity means 11

26 the average incremental rate of the counting process. For the special case of a Poisson process, the intensity function is the Poisson rate λ. The jump intensity function can be either independent of or dependent on the system dynamics. We call the former an uncontrolled communication scheduler and the latter a controlled communication scheduler. We use a jump-diffusion process to model the estimation error e(t), which evolves as a diffusion process most of the time and resets to the neighborhood of the origin when there are data arrivals from the network. The continuous-time LTI system can be estimated over the network if the error e(t) is bounded in a statistical sense. Our basic conclusions are roughly stated as follows: 1. When the intensity λ is constant, the continuous-time LTI system can be estimated in the mean square sense if λ > 2 max{r[eig(a)]}. The equation above means that the data communication has to be fast compared to real part of the least stable LTI system pole. This is in contrast to the minimum bit rate problem, in which all the unstable poles are relevant [34, 18]. 2. We give feedback mechanisms to communication schedulers by relating the estimation error to the intensity function λ( ), e.g., λ(e) = (e P e) k, for any P > 0 and k > 0. We prove that this controlled communication scheduler always gives enough data packets for LTI system estimation. 12

27 We consider a much richer class of communication schedulers in Chapter 4. Also we address issues such as data packet losses and communication delays in the same framework. Optimal communication schedulers We summarize the main results on optimal communication schedulers. The objective is to minimize both the average estimation error variance and the communication cost, i.e., the cost function takes the form of: cost = estimation error variance + communication load. In the discrete-time domain, the communication scheduler assigns a data sending probability u t at each time t. We optimize the estimation scheme with respect to local estimators, communication schedulers, and remote estimators. However, the emphasis is on optimal communication schedulers. We take a dynamic programming (DP) approach. In essence, we address a long-term average cost problem with an unbounded per-stage cost. Our basic conclusions are roughly stated as follows: 1. If the network does not drop data packets, a DP equation exactly solves the optimal communication scheduling problem. We develop an iterative numerical method to solve the optimal communication scheduler and calculate the optimal cost. The optimal communication scheduler is stationary and deterministic, and it takes the form of: 0, if the estimation error is in Ω u t = 1, otherwise, 13

28 where Ω is a set that can be pre-computed. 2. If the network drops data packet with probability p, we give both upper and lower bounds. The upper bound is related to the solution to a Lyapunov equation, while the lower bound is by recourse to results from the lossless network case. Bit and packet: a unified framework Consider the remote estimation scheme in Figure 1.3, in which the local node sends data packets to the remote estimator. The cardinality (the number of all possible codeword) of each packet is µ, and therefore the packet size is R b := log 2 µ bits. Packets are sent with probability 1 p, 0 p 1, and therefore the average packet rate is R = 1 p. For the remote estimation of a discrete-time LTI system (A, C), where A R n n, we have the following: 1. If integer µ > eig i (A) and p < 1 eig i (A) 2, 1 i n, the system can be estimated in the mean square sense over the channel, if and only if µ 2 > eig i (A) 1 (1 p) eig i (A) 2 1 p eig i (A) We observe that when the bit rate R b is small, increasing it effectively reduces the packet rate requirement. However, it becomes less effective when the packet size R b is large. We will unify some key results on data rate limitations in the literature (see Chapter 2). 14

29 1.4 Summary The dissertation embraces two issues on data communication scheduling methods: 1. We propose stochastic communication schedulers. Stochastic analysis methods are used to analyze the dynamics as jump-diffusion processes. The analysis methods are applicable to practical networks models. 2. We derive optimal stochastic communication schedulers that save network usage and improve estimation performance. Dynamic optimization techniques are applied to approach a long-term average cost problem with an unbounded per-stage cost function on a Borel state space. The results shed light on how to schedule data communication for NCSs. Another contribution is on the extension of data rate limitations. In the context of an estimation over networks problem, we derive conditions for the existence of estimators in terms of the bit rate and the packet rate. The conditions also shed light on how to allocate the data packet size, which is instrumental in NCS communication protocol design. In Chapter 2, we give a unified answer to both the bit rate and the packet rate requirements in the context of the estimation of discrete-time LTI systems over networks. In Chapter 3, we briefly summarize results on jump-diffusion processes, which provides analysis tools for Chapter 4. In Chapter 4, we propose stochastic communication schedulers for estimation over networks problems and analyze the estimator s stability properties. We consider plants with full and partial state measurements and pursue two objectives: 15

30 observability and finite network resource usage. We use counting processes such as the Poisson process to model the schedulers and use jump-diffusion processes to model estimator dynamics. The method based on infinitesimal generator allows us to analyze a large class of stochastic communication schedulers and various types of networks (e.g., networks with delays and networks with packet dropouts). In Chapter 5, we pursue optimal communication schedulers. For networks with no data dropouts, we solve an optimal communication problem via an average cost optimality equality (ACOE). To do that, we take advantage of some recurrent properties of the process and pose it as an essentially bounded per-stage cost problem. For general networks with a packet dropout probability p, we obtain both an upper bound and a lower bound on the optimal solution. The upper bound is obtained by solving an average cost optimality inequality (ACOI), and the lower bound is from an ACOE. In Chapter 6, we give some concluding remarks and directions for future work. 16

31 Chapter 2 Rate limitations This chapter is on data rate limitations, which include both the bit rate limitation and the packet rate limitation. The limitations are illustrated through the estimation of a discrete-time linear time-invariant (LTI) plant over networks. The local node sends data to the remote estimator, one packet per time step. Each packet contains the same fixed number of data bits. Due to network uncertainties, a packet may get lost with probability p (0 p 1), which results in an average packet rate of 1 p. We investigate whether it is feasible to construct estimators for the LTI systems over such communication channels. In Section 2.1, we state the main results, which reveal trade-off relations between packet size and required data transmission reliability. In Section 2.2, we discuss several special cases and then make connections to results in the literature. In Section 2.3, we list a few observations, which convince us to go from bits to packets. 17

32 2.1 Rate limitations for LTI systems In this section, we derive necessary and sufficient conditions for stability of a remote encoder-estimator scheme. For simplicity, we start with one-dimensional noiseless LTI systems, followed by systems with noise. Explicit encoder-estimator pairs are also constructed. The results are then extended to general LTI systems Problem setup We consider the estimation over networks (see Figure 2.1.1). The network is viewed as a digital channel that transmits one symbol per second from an alphabet (cardinality µ N). One symbol is packed in one data packet, resulting a sending PSfrag replacements bit rate of [47, 10]: R b := log 2 µ bps. Plant y t Encoder s t Network Dec/Est. Acknowledgment Figure 2.1. Estimation over networks We consider a general family of encoders, which may have infinite memory. In particular, a symbol s t is obtained by s t = E t (Y t, S t 1 ), t 0, (2.1) where E t ( ) is the encoding mapping, Y t := {y j j t} denotes the sequence of all measurements up to time t, and S t 1 := {s j j t 1} denotes the sequence of all symbols sent by the encoder up to time t 1. 18

33 Similarly, we consider a general family of decoders/estimators. In particular, the estimate ˆx t at time t is obtained by ˆx t = D t ( S t ), t 0, (2.2) where D t ( ) is a decoding mapping and S t := { s j j t} denotes the sequence of all symbols received by the decoder up to time t. For ideal channels, the symbols sent by the encoder always coincide with those received by the decoder, and therefore s t = s t, for all t 0. However, for noisy channels the symbols received may occasionally differ from the ones sent. We are interested in erasure channels, in which the symbols s t sent to the network take values in the set {1,, µ}, but the symbols s t received from the network take values in the set {1,, µ, ø}, where ø represents erasure. The symbol received differs from the symbol sent with probability p, independently and identically distributed (i.i.d.), and we have s t, with prob. 1 p s t = ø, with prob. p t 0. The encoder is informed when an erasure has occurred and can use this piece of information to encode subsequent symbols [10]. Channels may also introduce delays. For example, in lossless τ-delay channels the symbols are received τ time steps after they are sent, which corresponds to the following channel model, s t = s t τ, t 0. The following channel is the basic network model in this chapter: Definition 2.1 (1-delay p-erasure channel with feedback acknowledgment) A channel is called a 1-delay p-erasure channel with feedback acknowledgment if 19

34 it transfers one data packet (with cardinality µ) each time step, the output { s t } t 1 and the input {s t } t 0 satisfy, s t 1, with prob. 1 p s t = ø, with prob. p t 1, (2.3) and furthermore at time t the channel acknowledges the encoder whether the symbol s t 1 is received or not. For brevity, the channel is referred to as the erasure channel (2.3) in the rest of this chapter. The expected number of packets the channel successfully transfers per second is 1 p, which is defined to be the channel packet rate, R = 1 p. (2.4) One-dimensional noiseless LTI systems Consider one-dimensional discrete-time unstable noiseless systems x t+1 = Ax t, x 0 X 0, (2.5) y t = x t, where A > 1, x 0 is the only source of uncertainty, and X 0 is a closed interval 1. The initial set X 0 is known to both the encoder and the estimator. We consider estimation of (2.5) over the erasure channel (2.3). Assumption 2.1 The initial condition x 0 is uniformly distributed on X 0. 1 For unbounded X 0, one can use techniques based on entropy, as in [34]. 20

35 From Assumption 2.1, x t is uniformly distributed on A t X 0. Furthermore, for any subset B t A t X 0, x t B t implies that x t is uniformly distributed on B t. Under Assumption 2.1, it can be shown that, for any µ-quantizer Q µ : X Z µ, the infimum, inf Q µ max i Z µ min ˆx E [ x ˆx (Q µ (x)) m Q µ (x) = i] is achieved by a uniform quantization, where ˆx : Z µ X is an estimate of x. Without loss of generality, we further assume that encoders are uniform. Assumption 2.2 Encoders are uniform, i.e., the bounded set X is equally partitioned into µ subsets, each of which is assigned a unique codeword. Necessary and sufficient conditions We derive necessary and sufficient observability conditions for the estimation over networks problem. Stability is defined as boundedness of a signal in stochastic moment sense. Refer to [25] for general definitions on stochastic stabilities. Definition 2.2 (Stability in the m th moment) A discrete-time random process x t has initial distribution x t = x 0 such that E[ x 0 m ] < δ, for some δ > 0. The process x t is stable in the m th moment, if for any T > 0, there exists ɛ > 0, such that for any t > T, we have E[ x t m ] < ɛ, 21

36 where stands for the Euclidean norm. If in addition, lim E[ x t m ] = 0, t the process is asymptotically stable in the m th moment. Obervability is defined accordingly: Definition 2.3 (Observability in the m th moment) A linear system with the state x t is (resp. asymptotically) observable over the erasure channel (2.3) in the m th moment if there exists an encoder-decoder pair such that x t ˆx t is (resp. asymptotically) stable in the m th moment. When m = 2, it is mean-square (resp. asymptotically) observable. Theorem 2.1 For integer µ > A > 1, p [0, 1), and any m (0, + ), the system (2.5) is asymptotically observable in the m th moment over the erasure channel (2.3) if and only if p < µm A m µ m A m A m, or equivalently, µm > (1 p) A m 1 p A m. We interpret the theorem in Figure 2.2, for which we choose A = 2 and m = 2. The system is asymptotically observable in the second moment in the region above the curve. Note that when the drop rate is higher, larger packet sizes are required for observability. We save further comments to Section 2.3, where we will interpret from another viewpoint. Now we prove Theorem 2.1. Proof.[Theorem 2.1] Suppose that, at time t, the state is x t, the confidence estimation set is X t t (the minimum set in which x t lies, given the codewords 22

37 PSfrag replacements codeword cardinality µ observable unobservable drop rate p Figure 2.2. Graphical representation of Theorem 2.1. up to time t), and the best estimate is ˆx t. Assume µ is an positive integer (see Remark 2.1 for general cases). At time t + 1, if the codeword s t is not received, the confidence set becomes AX t t. Otherwise, the confidence set becomes one of the µ equal partitions of AX t t. We have that E [ x t+1 ˆx t+1 m ] = E [ A m x t ˆx t m lost] + E ( ( A = p A m + (1 p) µ [( ) m A x t ˆx t m received] µ ) m ) E [ x t ˆx t m ], from which we conclude that for asymptotic observability in the m th moment, it is necessary and sufficient that p A m + (1 p) ( ) m A < 1, (2.6) µ which is then converted to the inequalities in the theorem. Remark 2.1 It is conjectured that Theorem 2.1 is valid for non-integer µ. If µ = s/t is a rational number, where integers s and t are coprime, we can extend the proof above by considering t time steps instead of one time step. This result may be further extend to real numbers as rational numbers are dense on the real axis. 23

38 An encoder-estimator pair The encoder-estimator pair constitutes a synthesis proof for Theorem 2.1. Denote the centroid of the confidence interval X t as z t, the encoding precision as t, and the codeword as c t. Suppose the cardinality of the alphabet µ is an odd integer 2, and the alphabet is denoted by Z µ := { µ+1,, 0,, µ 1 }. We construct the 2 2 encoder-estimator pair recursively as follows: Encoder: 1. At time 0, initialize 0, z 0, and c 0 : 0 = max(x 0) min(x 0 ), µ z 0 = centroid of X 0, c 0 = At time t + 1, the sender gets the acknowledgment of whether c t is received, from which the encoder calculates the centroid z t+1 and the precision t+1 : A(z t + c t t ), if c t is received, z t+1 = Az t, otherwise, A t+1 = µ t, if c t is received, A t, otherwise. (2.7) 3. Also at time t + 1, the encoder finds the codeword c t+1 such that, x t+1 [z t+1 + (c t ) t+1, z t+1 + (c t ) t+1). 2 Otherwise, the representation needs to be slightly changed. 24

39 4. Finally, at time t + 1, the encoder sends c t+1 to the network. Estimator: 1. At time 0, fetch 0 and z 0 from the encoder and initialize ˆx 0 to be At time t + 1, the estimator may have received the codeword c t. The estimate is then calculated: z t + c t t, ˆx t = z t, if c t is received, otherwise, ˆx t+1 = Aˆx t. 3. Also at time t + 1, the decoder updates the centroid z t+1 and the estimation precision t : A(z t + c t t ), if c t is received, z t+1 = Az t, otherwise, A t+1 = µ t, if c t is received, A t, otherwise. The encoder-estimator pair above, z t and t can be synchronized on the encoder and the estimator. The estimation error, e t := x t ˆx t, satisfies ( 1 e t 2 t, 1 2 t], if c t is received, ( µ+1 2 t, µ 1 2 t], otherwise. Because e t is uniformly distributed, it is asymptotically stable in the m th moment if and only if t is. Note that the estimation precision t satisfies A t, with prob p, t+1 = A µ t, with prob 1-p. 25

40 For the m th moment stability, [( ) m A E[ m t+1 ] = E[( A t) m c t is lost] + E m t µ ( ( ) m ) A = p A m + (1 p) E[ m t ], µ from which the condition (2.6) is sufficient. ] c t is received One-dimensional LTI systems with noise Consider a one-dimensional LTI system with bounded random noise w t [ L w, L w ] and v t [ L v, L v ], x t+1 = Ax t + w t, x 0 X 0, (2.8) y t = x t + v t, where A > 1 and X 0 is a closed interval. The following theorem gives a result similar to Theorem 2.1: Theorem 2.2 For integer µ > A > 1, p [0, 1), and any m (0, + ), the system (2.8) is observable in the m th moment over the erasure channel (2.3) if and only if p < µm A m µ m A m A m, or equivalently, µm > (1 p) A m 1 p A m. The necessary part of the theorem can be derived as in the proof of Theorem 2.1. For the sufficient part, we construct the same encoder-estimator pair as in the noiseless case. Note now the centroid z t and precision t are updated 26

41 according to, A(z t + c t t ), if c t is received, z t+1 = Az t, otherwise, 1 t+1 = ( A µ t + 2 A L w + 2L v ), if c t is received, A t + 2 A L w + 2L v, otherwise, in which t is stable in the m th moment if (2.6) is satisfied. Remark 2.2 Bounded noise does not impose additional data rate requirements. This is because the noise terms affect the estimate uncertainty additively, while the data infusions from the new packet arrivals result in uncertainty reduction multiplicatively. However, the asymptotic convergence is lost Multi-dimensional LTI systems Consider an n-dimensional system with bounded random noise w t [ L 1 w, L 1 w] [ L n w, L n w] and v t [ L 1 v, L 1 v] [ L m v, L m v ], x t+1 = Ax t + w t, x 0 X 0, (2.9) y t = Cx t + v t, in which X 0 is a bounded set in R n, A R n n, C R m n, and (C, A) is an observable pair. The proof of the following theorem runs parallel to that of Theorems 2.1, but requires a coordinate transformation [57] and the construction of a local Kalman estimator (as in Chapter 5). The proof is omitted here. 27

42 Theorem 2.3 If integer µ > eig i (A) and p < 1 eig i (A) m, 1 i n, for any m (0, + ), the system (2.9) is observable in the m th moment over the erasure channel (2.3) if and only if µ m > eig i (A) 1 (1 p) eig i (A) m 1 p eig i (A) m. 2.2 Special cases In this section we apply the theorems to derive results that previously appeared in the literature Large packet sizes If the packet size is very large, i.e., µ, the condition in Theorem 2.3 becomes which is consistent with [62]. p < 1 max i eig i (A) m, (2.10) Lossless networks If the channel does not drop data packets, i.e., p = 0, the condition in Theorem 2.3 leads to µ > eig i (A), (2.11) eig i (A) 1 28

43 which is in parallel to the results from [34]. In Nair and Evans [34], with the same channel and information pattern, they consider the control of the LTI system: x t+1 = Ax t + Bu t, y t = Cx t, t 0, x t R n, u t R m, y t R l, with the pair (A, B) controllable and the pair (C, A) observable. A random initial condition x 0 is the only source of uncertainty. They prove necessary and sufficient conditions for stabilization over the network: R b > max{0, log 2 eig i (A) }, (2.12) eig i (A) which is obtained by taking logarithm of (2.11) Erasure networks When m approaches 0 from above, i.e., m 0, we obtain from Theorem 2.3 an almost sure 3 stability criterion, 1 R b > lim m 0 m log 2 eig i (A) 1 1 lim m 0 m log 2 eig i (A) 1 = = eig i (A) 1 = 1 1 p (1 p) eig i (A) m 1 p eig i (A) m (1 p) eig i (A) m 1 p eig i (A) m [ 1 lim m 0 m [log 2(1 p) log 2 (1 p eig i (A) m )] + log 2 eig i (A) eig i (A) 1 log 2 eig i (A), ] which is consistent with [56]. Tatikonda and Mitter [56] consider the erasure channel in (2.3). They provide a sufficient condition for the almost sure observ- 3 Stability with probability one. Refer to [25] for a definition. 29

44 ability of the system (2.9) if R b > 1 1 p eig i (A) max{0, log 2 eig i (A) }. (2.13) Although (2.13) is equivalent to (2.12) if p = 0, for p > 0, the inequality (2.13) is not sufficient for the existence of encoder-estimator that guarantees mean-square observability An example for rate limitations We consider an LTI system to illustrate data rate limitations, including the bit rate (R b ) and the packet rate (R). Example 2.1 Consider the estimation of the following LTI plant, x t+1 = 2 0 [ ] 0 x t + w t, y t = 1 1 x t + v t, (2.14) 2 where w t and v t are random variables uniformly distributed on bounded sets. The network is as in Definition When the channel does not drop packets (p = 0), for the system to have observability, the bit rate must satisfy (2.12), i.e., R b > log 2 + log 2 2 = 1.5 bps. 2. For all p [0, 1), to have almost sure observability, the data sending bit rate (i.e. the packet size) must satisfy (2.13), i.e., R b > 1 1 p (log 2 + log ) = 1 p = 1.5 R bps, where the last equality is from (2.4). 30

45 3. If p [0, 1 ) and the packet-size is fixed, to have observability in the second 4 moment, we obtain from Theorem 2.3 that the data sending bit rate (i.e., the packet size) must satisfy, R b > ( log 2 2 ( log 2 = p 1 4p + log 2 R 4R 3 + log 2 ) 1 p 1 2p R 2R 1 ) bps, which can be proved to be larger than 1.5 R observability). bps (as required for almost sure 2.3 From bits to packets In this section, we show that under certain conditions, the packet rate is the determining factor for both observability and communication channel performance Bit rate vs. packet rate We examine the roles of the bit rate and the packet rate in the observability of a linear plant over the channel (2.3). From Theorems , we observe a trade-off relation between the bit rate and the packet rate: a larger bit rate R b tolerates larger loss probability p, as long as p < 1 max i eig i (A) m ; a larger packet rate R, i.e., smaller dropout rate p, allows smaller sending bit rate R b. Consider the same one-dimensional case (Theorem 2.1) as in Figure 2.2 but with the appropriate coordinate transformation. Figure 2.3 depicts the observable and unobservable regions. The following observation can be made: 31

46 Remark 2.3 For a fixed packet rate R, once the bit rate R b is reasonably large compared to the system dynamics, increasing R b does little to expand the observable region, while increasing the packet rate is much more effective. PSfrag replacements Channel packet rate R unobservable observable Sending bit rate R b Figure 2.3. Packet rate vs. bit rate. We may reach the conclusion above from sensitivity analysis. From the conditions in Theorem 2.1, the slope of the curve in Figure 2.3, ( ) d µ m A m = m(1 A m )µ m 1, dµ µ m A m A m (µ m 1) 2 approaches zero as µ grows Packet rate and communication performance The number of packets is a major factor that determines the data traffic in a network. In most widely used communication protocols, the overhead of a packet is not reduced by decreasing the number of data-bits in the packet. The sending packet rate affects communication system performance in several ways, e.g., Higher data traffic may induce longer communication queuing delay, which is undesirable in real-time systems. 32