Networked Control Systems with Packet Delays and Losses

Proceedings of the 47th IEEE Conference on Decision and Control Cancun, Mexico, Dec 9-, 28 ThB6 Networed Control Systems with Pacet Delays and Losses C L Robinson Dept of Industrial and Enterprise Systems Engineering, and the Coordinated Science Lab, University of Illinois at Urbana-Champaign clrobnsn@uiucedu P R Kumar Dept of Electrical and Computer Engineering, and the Coordinated Science Lab, University of Illinois at Urbana-Champaign prumar@uiucedu Abstract We investigate the effect of pacet delays and pacet drops on networed control systems First we consider the problem of where to locate a controller or state estimator in a networ, and show that under a Long Pacets Assumption LPA) it is optimal to collocate it with the actuator We then show that under the LPA, stabilizability is only determined by the pacet drop probability and not the pacet delay probabilities We also consider a suboptimal state estimator without the LPA, based on inverting submatrices of the observability Krylov sequence I INTRODUCTION Contention for the medium, channel fading, and interference in networs, lead to pacet delays and losses Even though observations may be taen at regular instants, their arrivals after passage through the networ may be random since collision detection and avoidance algorithms use random bacoffs and delays, or they may even be dropped due to the losses in the wireless medium or collisions Hence, we address the issue of random pacet delays as well as pacet drops in networed control systems We study an LQG system by employing a Long Pacets Assumption LPA) [5], which allows pacets to be arbitrarily long, and in particular to contain a history of all past observations The LPA can be realized even without long pacets by having an encoder at the sensor and a decoder at the actuator, as shown in [5] We first address the question of where the control logic should be placed within the networ subject to random pacet delays and losses, and show that it is optimal to collocate the controller with the actuator This extends an earlier result for the case of pacet drops only [4], [5] Then we address the question of when such a system is stabilizable We show that the condition λ max A) 2, ) whereλ max A)ismagnitudeoftheeigenvalueofAwith the largest magnitude is necessary, and also sufficient when the inequality is strict This result shows that stabilizability under the LPA depends only on the loss probability and not the delay probabilities Thus the condition for stabilizability is essentially the same as in the pacet drop only case examined in [5], [8], [6], [9] It is interesting that under LPA the delay distribution only affects system performance and not stabilizability This is illustrated via simulations in Section IV The result has implications not only for a system with long pacets, but also, as mentioned above, for the encoder-decoder scheme in [5] which realizes the LPA but without long pacets The result on stabilizability not depending on delays but only on drop probability is analogous to that of [6], [7], where the estimation problem without the LPA is considered, and a similar independence of estimator stability on delays is shown The stabilizability condition is different in that case since the LPA does not hold We next analyze window based schemes without an LPA, as has been considered in [6], [7] We consider a family of suboptimal schemes and obtain upper bounds on pacet drop probability, similar to ), that are sufficient for their stability Useful references for networed control include [], [3] Pacet delays and drops result in generally intractable non-classical information patterns [2], [2] The effect of random sampling times on optimal controller design [2], state estimation [2], [3] and overall system performance [8] have been considered Pacet delays are considered in [], with delay assumed to be less than one sampling period Another approach is to focus on eliminating the effect of random delay In [], abufferismaintainedatthereceivingendofthechannel for randomly delayed pacets, which releases them at regular intervals A similar actuator buffer has actually been deployed in [4] The condition ) is studied visa-vis pacet drops in [7], [8], [9] II CONTROLLER PLACEMENT We begin by determining where to locate the controller; see Fig We show that placing it on a path with best delay characteristics is optimal Let q ) be the probability mass function for delay on any lin j Pacets are dropped with probability t=qt) Delays of pacets on lins are assumed iid Letπ hg be the path of a pacet from a node h to node g The path delay is the sum of the lin delays on the path, with probability distribution denoted by F πhg We say that distribution F dominates F 2 if F D) 978--4244-324-3/8/$25 28 IEEE 462 Authorized licensed use limited to: The Ohio State University Downloaded on July 9,2 at 2:28:36 UTC from IEEE Xplore Restrictions apply

47th IEEE CDC, Cancun, Mexico, Dec 9-, 28 ThB6 S Sensor Plant C Actuator Fig The controller is at node C, andπ is a minimal path SA F 2 D) DApathbetweennodeshand gwiththe best delay characteristics, ie, one whose delay distribution dominates those of all other paths between h and g, if one exists, is referred to as a shortest path and is denoted byπ ; see Figure hg Lemma 2: Suppose the same pacet is sent along two chains of nodesc andc 2, as in Fig 2 Under the LPA, the information at each node inc with more nodes is dominated by that at a corresponding node in C 2 C C2 S S Fig 2 Two chains of nodes with source node denoted by node S Proof: Node A onc at the same hop distance from the source S as B see Fig 2, has F π = F π ) The SA SB information arrival processes at these two nodes can be stochastically coupled Since pacets will be further delayed between nodes A and C inc, the information at node B stochastically dominates that at C Corollary 22: Under the LPA there is an optimal controller placement that is on pathπ SA Theorem 23: Placing the controller at the actuator is optimal Proof: Consider Case with controller located at node i π, and Case 2 where it is located at A We SA stochastically couple the cases so that events of pacets from i reaching A are identical Hence, using LPA, node A can receive the same observation information in both cases However, the controller located at A additionally has the history of all implemented controls III SYSTEM AND PACKET DELAY MODEL Consider the observable and controllable system: x + = Ax +Bu +w, y j = C j x j +v j, A B z j = {y j,y j,y } 2) Noises w and v j are zero mean iid Gaussian processes with covariances Σ w and Σ v Pacet delays, x, {w } and {v } are mutually independent y j is the state observation made by the sensor at time j z j is the i A C transmitted observation, and under the LPA is comprised of all state observations{y : j} The sensor taes and transmits observations at each sampling instant Control actions are implemented every s sample instants, and are held constant during the intermediate interval These are called actuation instants Transmitted observations are subject to delay Since control actions are only implemented at actuation instants, a delayed measurement can arrive at any time, but can only be used at the next actuation instant We now describe the probabilistic model for the endto-end pacet delay; see Figure 3 A pacet transmitted at time j will contain z j under the LPA and is subject to delay There is no guarantee that pacets will arrive in order Delays of individual pacets are iid :=ProbPacet transmitted at arrives at +i) :=ProbPacet is dropped)= := P [ Pacet delay i ] = + j=i p j I) :=Set of observations nown to controller at is the information set α) := age of the information set= i, if the latest pacet to have arrived before actuation instant is z i Note thati)={y j : j α)} Π α ) Pα)=α) Note thatπ α lim Π α ) exists α DenoteΠ {Π,Π,} With a) below representing pacet α arriving by, and b) below pacets{ α+, α+2,,} that did not arrive by time, see Figure 3, we can write: a) b) { }} {{ }} { Π α )= p α+ ) p p 2 p 3 p α forα, forα> Sensor Estimator p α) s p p 2 p 3 p 4 p 5 Actuation instant Sample instant Fig 3 State observations occur at sample instants Control actions are computed and implemented at actuation instants Actuation instants are separated by s sample instants Sensor observations are sent over the networ and incur a delay i with probability, or are dropped with probability = p p 2 p 3 p 4 p 5 463 Authorized licensed use limited to: The Ohio State University Downloaded on July 9,2 at 2:28:36 UTC from IEEE Xplore Restrictions apply

47th IEEE CDC, Cancun, Mexico, Dec 9-, 28 ThB6 IV NECESSARY CONDITION FOR BOUNDED ESTIMATION ERROR COVARIANCE We address boundedness of the quadratic cost: N J = limsup N N E x Qx +u Ru = Denoting byσ x the state estimation error covariance at, which is random because of the randomness in obtaining measurements, we can write N J = limsup N N E ˆx Qˆx +u Ru +TrΣ x Q)), = So we need only study boundedness of E [ Σ x ] A State Estimation: Kalman Filter The Time update is, with the usual notation: ˆx + = Aˆx +Bu, 3) Σ x + = AΣ x A +Σ w, 4) The Measurement update is: K + = Σ x + C CΣ x + C +Σ v), 5) ˆx + + = ˆx + +K + y cˆx + ), 6) Σ x + + = I K + C)Σ x +, 7) At each actuation sample instant, 3) and 4) are used to update the system state estimate Iterating D steps, D Σ x +D = ADΣ x A D + A i Σ w A i 8) If there is no pacet loss or delay, then since [A,Σ w 2 ] is controllable and [A, C] detectable, the error covariance converges to a positive definite limit, which we denote by Σ x For simplicity we assume that the system is started with Σ x = Σ x Under the LPA, whenever a pacet z j arrives, the estimation error covarianceσ x j j reverts toσ x Hence it is the durations of the excursions fromσ x that determine boundedness B Bounded Estimation Error Covariance The expected estimation error covariance is E [ Σ x] = Π iσ x +i Theorem 4: ) is necessary for bounded cost Proof: Π α = α p j) α j= p j) So, using 8), E [ Σ x] = = Π i Σ x +i i Π i A i Σ x A i + A j Σ w A j 9) i i p j p j A i Σ x A i j= If v is the eigenvector of A corresponding to the eigenvalue of A with the largest magnitude,λ max A), then: v E [ Σ x] v i i v Σ x v) p j p j λ max A) 2i We now use the ratio test, noting that lim i = : i+ j= p ) ) j pi+2 λmax A) 2i+2 i i j= p ) ) j pi+ λmax A) 2i ) + pi+2 = limsup λ max A) 2 i + ) = λ max A) 2, establishing the necessity of ) Theorem 42: A sufficient condition for boundedness of E [ Σ x] is < λ max A) 2 Proof: Becauseγ I Σ x γ 2I, and the same is true for A i Σ W A i, we only need to consider boundedness of: E [ Σ x] = j= Π i Σ x +i γ 3 Π i A i A i i γ 3 p j λ max A) 2i, where we have upper bounded several terms by By the ratio test, a sufficient condition for stability is: ) i+ λmax A) 2i+2 i j= p j j= i j= p j) λmax A) 2i < λ max A) 2 < Hence stabilizability under LPA only depends on pacet drop probability and system dynamics Performance is adversely affected by larger delay, but not stabilizability This has potential design implications One can set the critical number of delivery attempts at the transport layer, similar to the MAC layer retry limit in IEEE 82, so as to meet a desired The performance of several specific delay distributions in Figure 4 is illustrated in Figures 5, 6 and 7 a) Uniform b) Exponential c) Positve Fig 4 The drop probability,, is chosen to be the same The density function, is chosen so to satisfy + = 464 Authorized licensed use limited to: The Ohio State University Downloaded on July 9,2 at 2:28:36 UTC from IEEE Xplore Restrictions apply

47th IEEE CDC, Cancun, Mexico, Dec 9-, 28 ThB6 4 Pacets Dropped After Sample Instants 6 λ max A) = 2 3 5 λ max A) = 2 4 2 3 4 5 6 7 Cost 3 3 25 2 Pacets Dropped After 2 Sample Instants 2 Cost 5 5 2 3 4 5 6 7 8 Pacet Drop Probability 2 8 6 2 3 4 5 6 7 Pacets Dropped After 3 Sample Instants Single Observation Long Pacets Fig 7 Each curve represents a different system Pacets are delayed with uniform probability Figure 4a)), and dropped when delay exceeds 3 The stability upper bound on p Dros indicated by the vertical asymptotes The upper curve represents a system with a largest eigenvalue of 2 and hence a stability bound of = 69 The lower curve is a system with largest eigenvalue as and stability bound of = 83 4 2 2 3 4 5 6 7 V STABILITY SUFFICIENT CONDITION WITHOUT LPA We consider a suboptimal scheme without LPA Fig 5 Pacets are delivered with uniform probability of delay as shown in Fig 4a) In the top figure, pacets delayed by more than sample instants are dropped In the second and third figures the delay threshold is 2 and 3, respectively The x-axis is the pacet drop probability The lower curve in each is the cost under LPA, and the upper curve that for system which only transmits a single observation Notice that the cost diverges at the same pacet drop probability in each figure Cost 2 5 5 Uniform Exponential Positive Bound 2 3 4 5 6 7 Pacet Drop Probability Fig 6 Pacets delayed 3 time steps are dropped The lowest curve is the cost for exponential pacet delay distribution Figure 4b)), the middle curve for uniform distribution Figure 4a)), and upper curve for linear distribution Figure 4c)) A A Sub-Optimal Estimation Scheme For an n-dimensional observable system, n consecutive observations yield a state estimate with bounded error covariance [9], denoted Σ x We consider a suboptimal filter that uses only the most recent set of n consecutive measurements that have arrived Open loop predictionisdoneinbetweensuchbatchesofnormore consecutive observation arrivals Denote by the elapsed time since the most recent time at which n consecutive pacets were delivered and the current time Denote this set of n observations as: Y) = { y j : n+ j } The resulting estimation error covariance at, ˆΣ x is: ˆΣ x = A n Σ n x A + A j Σ w A j being random, we compute the expectation E[E[ˆΣ x ]] Theorem 5: The expected estimation error covariance is bounded if lim inf p p + > λ max A) 2 ) Proof: If pacets n+, n+2,, ) are the latest n consecutive pacets to arrive before, then pacet + should have not arrived by ; see Fig 8 465 Authorized licensed use limited to: The Ohio State University Downloaded on July 9,2 at 2:28:36 UTC from IEEE Xplore Restrictions apply

47th IEEE CDC, Cancun, Mexico, Dec 9-, 28 ThB6 p { }} { n+ n p n p n Fig 8 The Figure illustrates an event where n consecutive pacets arrive before time The pacets which arrive may do so at anytime between their transmission time and time The potential arrival times are represented by the shaded area The sequence of consecutive pacets is broen by a pacet being delayed with probability p n This non-arrival occurs with probability p n Hence, [ ] Pacets n P n+, n+2,, ) are the last n consecutive ones to arrive before +n +n 2 p n ) Using ) and 8) we can compute an upper bound +n +n 2 E[ Σ x ] p n = n A ˆΣ x A + A j Σ w A j It is sufficient to consider the boundedness of +n +n 2 p n A A n) = +n +n +n p n A A n) = +n n = p n A A n) = } {{ } p n A A n) = p n λ max A) 2 = For boundedness, by the ratio test it is sufficient if p n +λ max A) 2+2 < p n λ max A) 2 This is assured if p p n ) < λ max A) 2 2) Interestingly, the condition is independent of n For a geometric delay p = p) p and p = p) the sufficient condition is p> λ max This is illustrated A) 2 in Figure 9 as theγ= curve B An improved suboptimal scheme An improved estimator can use any n observations from n+γ consecutive observations Denote byγthe largest integer such that any r observations drawn from {y r γ+,y r γ+2,,y } yield an estimate of x with boundederrorcovariance,andcall anestimationepoch, γ the number of allowable misses, and the estimator an allowable misses estimator Theorem 52: With γ allowable misses, a state estimate with bounded error covariance can be formed if lim inf p p > λ max A) 2 ) γ+ 3) Proof: The set of observations used at is Y ) {y j : r γ+ j and y j delivered} 4) Define the oldest possible observation in Y ) as r γ+ Say that Y ) is full if it is missing exactly γ observations in the interval n γ+ j For to be the most recent estimation epoch, clearly pacet y + should have been dropped, and in [ r γ+, ] there need to beγdrops Hence, [ ] is the most P recent epoch before ) r+γ p γ p p p K p γ+ }{{}, } {{ } γ missing from r+γ y + missing for a sufficiently large constant K Substituting j= : E[Σ x ] K p γ+ j j AjΣ x Y) A j + A i Σ w A i, It is enough to consider conditions for boundedness of p γ+ j A j A j) p γ+ λ j max A) 2j 5) By the ratio test, this is bounded if j p γ+ j+ λ maxa) 2j+) p γ+ λ j max A) 2j p ) j j p j <, ie, if < ) γ+ λ max A) 2 466 Authorized licensed use limited to: The Ohio State University Downloaded on July 9,2 at 2:28:36 UTC from IEEE Xplore Restrictions apply

47th IEEE CDC, Cancun, Mexico, Dec 9-, 28 ThB6 For a geometric delay distribution, the sufficient condition is p> γ+ This condition is illustrated in Figure 9 for different values of the term γ+) λ max A) 2 ) Fig 9 Illustration of Thm 52 for a geometric delay distribution for different values of the term γ+) The upper curve γ = ) corresponds to Thm 5 C Existence of γ For random A, usuallyγ For example,γ=6 for 655 5853 899 847 626 2238 9593 2543 A=, 6) 9 753 5472 843 4984 255 386 2435 C=[ ] throughout) Systems can have largeγ For example, A hasγ>7, but A 2 yieldsγ=: A = 2 2 3, and A 2= VI CONCLUSIONS 2 2 3 We have considered the effect of pacet losses and delay on networed control system stabilizability under a Long Pacets Assumption We have established an optimal controller location and obtained a necessary condition for stability which is sufficient if the inequality is strict The latter depends only on the drop probability and not the delay probability We have also considered a sub-optimal scheme that may possibly be of interest ACKNOWLEDGMENT REFERENCES [] Special Issue: Technology of Networed Control Systems Proceedings of the IEEE, Jan 27 [2] M Adès, P E Caines, and R P Malhamé Stochastic optimal control under poisson-distributed observations IEEE Transactions on Automatic Control, 45), January 2 [3] M Chow and Y Tipsuwan Networ-based control systems: A tutorial In The 27 th Annual Conference of the IEEE Industrial Electronics Society, pages 593 62, 2 [4] S Graham Issues in the convergence of control with communication and computation PhD thesis, University of Illinois at Urbana- Champaign, 24 [5] V Gupta, B Hassibi, and R M Murray Optimal LQG control across pacet-dropping lins Systems & Control Letters, 566):439 446, June 27 [6] OCImer, SYusel,andTBasar Optimalcontrolofdynamical systems over unreliable communication lins In NOLCOS, 24 [7] OCImer, SYusel,andTBasar Optimalcontrolofdynamical systems over unreliable communication lins Automatica, June 26 [8] A Khan, N Agarwal, D Tilbury, and J Moyne The impact of random device processing delays on networed control system performance In 42 nd Annual Allerton Conference on Communication, Control, and Computing, 24 [9] P R Kumar and P Varaiya Stochastic Systems: Estimation, Identification and Adaptive Control, volume Englewood Cliffs, 986 [] R Luc and A Ray An observer-based compensator for distributed delays In Automatica, volume 26, pages 93 98, 99 [] J Nilsson, B Bernhardsson, and B Wittenmar Stochastic analysis and control of real-time systems with random time delays Automatica, 34):57 64, 998 [2] C H Papadimitriou and J Tsitsilis Intractable problems in control theory SIAM Journal on Control and Optimization, 244):639 654, July 986 [3] I B Rhodes and D L Snyder Estimation and control performance for space-time point-process observations In IEEE Transactions on Automatic Control, volume AC-22, pages 338 346, June 977 [4] C L Robinson and P R Kumar Control over networs of unreliable lins: Location of controllers, control laws and bounds on performance In Proceedings of Control over Communication Channels ConCom), Cyprus, April 27 [5] C L Robinson and P R Kumar Optimizing controller location in networed control systems with pacet drops IEEE Journal on Selected Areas in Communications, 264):66 67, May 28 [6] L Schenato Optimal sensor fusion for distributed sensors subject to random delay and pacet loss In Proc of the IEEE Conf on Decision and Control, New Orleans, December 27 [7] L Schenato Optimal estimation in networed control systems subject to random delay and pacet drop IEEE Trans on Automatic Control, 535):33 37, 28 [8] P Seiler and R Sengupta Analysis of communication losses in vehicle control problems In Proc of the American Controls Conf, June 2 [9] B Sinopoli, L Schenato, M Franceschetti, K Poolla, M I Jordan, and S S Sastry Kalman filtering with intermittent observations In IEEE Transactions on Automatic Control, volume 49, 24 [2] D L Snyder and P M Fishman How to trac a swarm of fireflies by observing their flashes In IEEE Transactions on Information Theory, 975 [2] H S Witsenhausen A counter example in stochastic optimal control Siam J Control, 6:3 47, 968 This material is based upon wor partially supported by USARO under Contract Nos W9NF-8-- 238 and W-9-NF-7287, NSF under Contract Nos ECCS-764, CNS-7-2992, CNS-626584, CNS-5-9535 and CCR-32576, and AFOSR under Contract No F4962-2--27 467 Authorized licensed use limited to: The Ohio State University Downloaded on July 9,2 at 2:28:36 UTC from IEEE Xplore Restrictions apply