Positive Markov Jump Linear Systems (PMJLS) with applications

Positive Markov Jump Linear Systems (PMJLS) with applications P. Bolzern, P. Colaneri DEIB, Politecnico di Milano - Italy December 12, 2015

Summary Positive Markov Jump Linear Systems Mean stability Input-output norms State-feedback design Dual switching control design Application example Conclusions

Positive Linear Systems (PLS) Nonnegative initial state + Nonnegative input nonnegative state and output t 0 Monotonicity property Co-positive Lyapunov functions, e.g. v(x)=p x with positive p Linear I/O performance measures Several applications: Compartmental models, Economics, Ecology, Transportation networks,... 1 A. Berman, R.J. Plemmons Nonnegative Matrices in the Mathematical Sciences, SIAM, 1994. 2 L. Farina, S. Rinaldi Positive Linear Systems: Theory and Applications, Wiley, 2000. 3 T. Kaczorek Positive 1D and 2D Systems, Springer, 2002. 4 W. Haddad, V. Chellaboina, Q. Hui Nonnegative and Compartmental Dynamical Systems, Princeton Univ. Press, 2010.

Markov Jump Linear Systems (MJLS) Dynamical systems subject to random jumps within a finite set of linear models Jumps modeled by a Markov chain (random failures, sudden change of parameters, topology switches,...) Mean square stability associated with quadratic Lyapunov functions Mean square I/O performance measures 1 O.L.V. Costa, M.D. Fragoso, R.P. Marquez Discrete-Time Markov Jump Linear Systems, Springer, 2005. 2 E.K. Boukas Stochastic Switching Systems, Birkhauser, 2005. 3 O.L.V. Costa, M.D. Fragoso, M.G. Todorov Continuous-Time Markov Jump Linear Systems, Springer, 2013.

Positive Markov Jump Linear Systems (PMJLS) Positive linear systems subject to random jumps, modeled by a Markov chain Mean stability associated with co-positive linear Lyapunov functions Mean I/O linear performance measures Most analysis and design problems addressable through Linear Programming 1 P. Bolzern, P. Colaneri Positive Markov Jump Linear Systems to appear in Foundations and Trends in Systems and Control, 2015.

PMJLS - Motivating applications Compartmental models with switching parameters Power allocation in telecommunication networks Consensus problems with switching topology Epidemiological models

Further references on PMJLS 1 M. Ait Rami, J. Shamma Hybrid positive systems subject to Markovian switching, 3rd IFAC Conf. on Analysis and Design of Hybrid Systems, Zaragoza, Spain, 2009. 2 P. Bolzern, P. Colaneri, G. De Nicolao Stabilization via switching of positive Markov jump linear systems 53rd IEEE Conf. on Decision and Control, Los Angeles, USA, 2359-2364, 2014. 3 P. Bolzern, P. Colaneri, G. De Nicolao Stochastic stability of positive Markov jump linear systems Automatica, 50, 1181-1187, 2014. 4 J. Zhang, Z. Han, F. Zhu Stochastic stability and stabilization of positive systems with Markovian jump parameters Nonlinear Analysis: Hybrid Systems, 12, 147-155, 2014. 5 J. C. Geromel, G. S. Daecto, P. Colaneri Minimax control of Markov jump linear systems to appear in Int. J. of Adaptive Control and Signal Processing, 2015. 6 P. Colaneri, P. Bolzern, J. C. Geromel, G. S. Daecto State-feedback control of positive switching systems with Markovian jumps to appear in Optimization and Applications in Control and Data Science, Springer, 2016.

Notation The set of nonnegative real numbers is R + A matrix with nonnegative entries is nonnegative, M 0 A matrix with nonnegative entries and nonzero is positive, M >0 A matrix with positive entries is strictly positive, M 0 A square matrix with nonnegative off-diagonal entries is a Metzler matrix G N is the set of N N Metzler matrices with each row summing to 0 1 n is a column vector of size n with all entries equal to 1 e k is the k-th column of the identity matrix

Positive Markov Jump Linear Systems (PMJLS) Consider the system ẋ(t) = A θ(t) x(t)+b θ(t) w(t), x(0)=x 0 z(t) = C θ(t) x(t)+d θ(t) w(t) x(t) R n + is the state w(t) R n w + is a nonnegative deterministic disturbance z(t) R n z + is the performance output θ(t) N ={1,2,...,N} is a Markov chain with transition rate matrix (TRM) Λ=[λ ij ] A i are Metzler, B i, C i, D i are nonnegative matrices, i N

Markov chain The entries of the TRM Λ=[λ ij ] are defined as Pr{θ(t+h)=j θ(t)=i}=λ ij h+o(h), i j λ ii = N j=1,j i λ ij, Λ G N The probability distribution π(t) satisfies π(t)=λ π(t) If Λ is irreducible, π is the unique stationary probability distribution If π(0)= π, the process θ(t) is stationary and ergodic

Time evolution of the mean It results that E[x(t)]=m x (t)= N s i (t) i=1 with Now let s i (t)=e[x(t) θ(t)=i] π i, i N s(t)=vec{s i (t)} R Nn +

Time evolution of the mean The vector s(t) and E[z(t)]=m z (t) are the state and the output of the deterministic positive system ṡ(t) = m z (t) = Ãs(t)+ Bw(t) Cs(t)+ Dw(t) Ã=diag{A i }+Λ I n, B = col {B i π i } i i C = row i {C i }, N D = D i π i i=1 with associated mean transfer function G(s)= C(sI Ã) 1 B+ D

Mean stability Definition The PMJLS is M-stable if, for w(t)=0, it holds that lim E[x(t)]=0, x(0), π(0) t This definition makes sense only thanks to positivity It is equivalent to convergence to zero of the first moment E[ x(t) ] It is equivalent to the boundedness of E[ 0 x(t)dt]

Criteria for M-stability Proposition 1 M-stability of a PMJLS is equivalent to any of the following conditions: (i) Matrix Ã is Hurwitz (ii) There exist vectors s i 0 such that A i s i + N j=1 λ ji s j 0, i N (iii) There exist vectors p i 0 such that A i p i + N j=1 λ ij p j 0, i N Note that (ii) and (iii) are Linear Programming (LP) feasibility problems

L -induced norm Let w L (bounded disturbance). The L -induced norm of a PMJLS is defined as J := sup k,t 0 ek sup m z(t) w L,w>0 sup k,t 0 ek w(t) It is a measure of average peak-to-peak disturbance attenuation It can be shown that J = G(0) =max k e k G(0)1 nw

Criterion for guaranteed L -induced norm Proposition 2 The PMJLS is M-stable with J < ρ if and only if there exist vectors s i 0 such that A i s i + N j=1 N i=1 λ ji s j + π i B i 1 nw 0, i N (C i s i + π i D i 1 nw ) 1 nz ρ The worst-case disturbance is w(t)=1 nw,t 0 The optimal performance is obtained by minimizing ρ through LP

L 1 -induced norm Let w L 1 (integrable disturbance). The L 1 -induced norm of a PMJLS is defined as 0 1 n J 1 := sup z m z (t)dt w L 1,w>0 0 1 n w w(t)dt It is a measure of average integral disturbance attenuation It can be shown that J 1 = G(0) 1 =max k 1 n w G(0)ek

Criterion for guaranteed L 1 -induced norm Proposition 3 The PMJLS is M-stable with J 1 < ρ if and only if there exist vectors p i 0 such that A i p i + N i=1 N j=1 λ ij p j +C i 1 nz ) (B i p i +Di 1 nz π i 0, i N ρ1 nw The worst-case disturbance is w(t)=δ(t)e k, being k the worst input channel The optimal performance is obtained by minimizing ρ through LP

Summary on M-stability and I/O norms Given the mean transfer function G(s)= C(sI Ã) 1 B+ D M-stability is related to stability of Ã I/O norms are related to the norm of the dc-gain G(0)= D CÃ 1 B In particular, J = G(0), J 1 = G(0) 1 When n w =n z =1, it follows that J =J 1 Checking stability and guaranteed I/O performance can be performed via Linear Programming tools

Positive parameter perturbations Apply perturbations to the entries of matrices A i, B i, C i, D i The spectral abscissa of Ã is a monotonically increasing function of the entries of A i No positive perturbation of A i can stabilize an unstable PMJLS The dc-gain G(0)= D CÃ 1 B is positive and its entries are monotonically increasing functions of the entries of A i, B i, C i, D i (as long as stability is preserved) No positive perturbation of any system matrix can improve the value of J or J 1

Control of PMJLS - Linear state feedback ẋ(t) = A θ(t) x(t)+g θ(t) u(t)+b θ(t) w(t) z(t) = C θ(t) x(t)+h θ(t) u(t)+d θ(t) w(t) u(t) = K θ(t) x(t) Problem: Design mode-dependent gains K i (or a mode-independent gain K) such that the closed-loop system is positive and M-stable (and achieves a prescribed I/O performance) θ w PMJLS x z u K

Control of PMJLS - Dual switching control ẋ(t) = A γ(t) θ(t) x(t)+bγ(t) θ(t) w(t) z(t) = C γ(t) θ(t) x(t)+dγ(t) θ(t) w(t) γ(t) = f(x(t),θ(t)), γ(t) M ={1,2,...,M} Problem: Design a mode-dependent switching law f(x,θ) (or a mode-independent law f(x)) such that the closed-loop system is M-stable (and achieves a prescribed I/O performance) θ w z S γ PMJLS x

Control of PMJLS - Co-design ẋ(t) = A γ(t) γ(t) θ(t) x(t)+gθ(t) u(t)+bγ(t) θ(t) w(t) z(t) = C γ(t) θ(t) x(t)+hγ(t) θ(t) u(t)+dγ(t) θ(t) w(t) u(t) = K γ(t) θ(t) x(t), γ(t)=f(x(t),θ(t)), γ(t) M Problem: Jointly design the linear feedback gains K i and the switching law f(x,θ) θ w z S γ PMJLS u K x

Control of PMJLS - Linear state feedback The closed-loop system under linear state feedback is ẋ(t) = (A θ(t) +G θ(t) K θ(t) )x(t)+b θ(t) w(t) z(t) = (C θ(t) +H θ(t) K θ(t) )x(t)+d θ(t) w(t) If u(t)=k θ(t) x(t) is constrained to be nonnegative, the gains K i must be nonnegative This induces positive perturbations of the entries of A i, C i No unstable open-loop PMJLS can be stabilized by nonnegative u(t) No positive perturbation of M-stabilizing gains can improve the costs J and J 1 Only the positivity of the closed-loop PMJLS will be required

Control of PMJLS - Linear state feedback Available results Mode dependent gains Full parametrization (via LP) of all M-stabilizing gains all gains ensuring J < ρ all gains ensuring J 1 < ρ with single disturbance Sufficient conditions for the existence of a gain ensuring J 1 < ρ for multiple disturbances Mode independent gains Same as before, but via nonlinear inequalities (LP for scalar input)

Control of PMJLS - Linear state feedback Mode dependent L -induced control Theorem 4 There exist K i, i N, such that the closed-loop system is positive and M-stable with J < ρ if and only if there exist vectors s i 0 and vectors h p i such that, for all i N, r =1,2,...,n, p =1,2,...,n, q =1,2,...,n z, The gains K i are given by n A i s i +G i h p N i + λ ji s j +[ π] i B i 1 nw 0 p=1 j=1 ( ) N n C i s i +H i h p i +[ π] i D i 1 nw ρ1 nz i=1 p=1 e r G i h p i +[A] rp e ps i e q H ih p i +[C] qp e p s i 0 0, r p K i e p =(e p s i) 1 h p i, p =1,2,...,n

Control of PMJLS - Linear state feedback Mode independent L -induced control Theorem 5 There exists K such that the closed-loop system is positive and M-stable with J < ρ if and only if there exist vectors s i 0 and vectors h p i, hp i such that, for all i N, r =1,2,...,n, p =1,2,...,n, q =1,2,...,n z, n A i s i +G i h p N i + λ ji s j +[ π] i B i 1 nw 0 p=1 j=1 ( ) N n C i s i +H i h p i +[ π] i D i 1 nw ρ1 nz i=1 p=1 e r G ih p i +[A] rp e p s i e qh i h p i +[C] qp e ps i 0 0, r p h p i h p i All admissible gains K must satisfy (e p s i) 1 h p i Ke p (e p s i) 1 h p i, i N,p =1,2,...,n

Control of PMJLS - Linear state feedback Open problems Full parametrization of L 1 -induced controllers with multiple disturbances LP tools for design of mode-independent controllers with multiple control inputs Possible constraints on u(t) Output feedback controller design

Control of PMJLS - Dual switching control Available results Sufficient conditions for mode-dependent and mode-independent M-stabilization and L 1 -induced control Open problems L -induced control design Co-design of linear feedback gains and switching law

Control of PMJLS - Dual switching control Mode dependent M-stabilization Theorem 6 Assume that there exist an irreducible matrix Φ=[ϕ rs ] G M and vectors p j i 0 such that, i N,j M, (p j i ) A j N i + λ ik (p j M k ) + ϕ jr (p r i ) 0 k=1 r=1 Then the state-feedback switching law given by makes the closed-loop system M-stable. γ =g(x,θ)= argmin r (p r θ ) x

Compartmental systems Compartmental systems are composed of N c interconnected reservoirs, exchanging flows of a common resource. Compartment i is modeled by ẋ i (t)= α i x i (t)+v i (t), y i (t)=β i x i (t), β i α i The total inflow to compartment i is described by v i (t)= N c j=1 γ ji y j (t)+δ i w(t), 0 γ ji 1, δ i 0 N c i=1 γ ji 1, N c i=1 δ i =1 The overall system is a Positive Linear System.

Stochastic compartmental systems If some parameters are subject to sudden random changes within finite sets, the system becomes a PMJLS, modeled as ẋ(t)=a θ(t) x(t)+b θ(t) w(t) The model is completed by the definition of the performance output z(t)=c θ(t) x(t) and the knowledge of the TRM Λ=[λ ij ] G N, where N is the number of modes.

Example - 4-tanks compartmental system = = = =

Example - 4-tanks compartmental system Parameters Assume N =2, and α 1 = α 2 = α 3 = α 4 =1, β 1 = β 3 = β 4 =1 γ 13 = γ 14 =0.5, γ 21 = γ 23 = γ 24 =0.33 γ 33 =0.2, γ 34 =0.4, γ 42 =0.5, δ 1 = δ 2 =0.5 β 2 = 0.8, if θ =1 β 2 = 0, if θ =2 [ ] 0.1 0.1 Λ= 0.5 0.5

Example - 4-tanks compartmental system Open-loop impulse response 1 1 x 1 0.8 0.6 0.4 0.2 0 0 5 10 t 0.2 0.15 x 2 0.8 0.6 0.4 0.2 0 0 5 10 t 0.2 0.15 x 3 0.1 x 4 0.1 0.05 0.05 0 0 5 10 t 0 0 5 10 t

Example - 4-tanks compartmental system Open-loop step response 1 1 x 1 0.8 0.6 0.4 0.2 x 2 0.8 0.6 0.4 0.2 0 0 5 10 t 1 0.8 0 0 5 10 t 1 0.8 x 3 0.6 0.4 0.2 x 4 0.6 0.4 0.2 0 0 5 10 t 0 0 5 10 t

Example - 4-tanks compartmental system Linear state feedback design Suppose that the system can be controlled by exchanging the flow u(t)=k θ(t) x(t) from tank 1 to any other compartment. Configuration A: exchange flow between tanks 1 and 2 Configuration B: exchange flow between tanks 1 and 3 Configuration C: exchange flow between tanks 1 and 4 For any configuration, design K i, i =1,2 so as to minimize 0 1 J 1 = sup nm x (t)dt sup k,t 0 0, J = sup e k m x (t) w L 1 w(t)dt w L sup t 0 w(t)

Example - 4-tanks compartmental system Configuration A = = = = =

Example - 4-tanks compartmental system Configuration B = = = = =

Example - 4-tanks compartmental system Configuration C = = = = =

Example - 4-tanks compartmental system Results of linear state feedback design Open-loop: J 1 =2.5164, J =0.7388 Configuration A: J 1 =2.2981, J =0.6727 Configuration B: J 1 =2.1079, J =0.7139 Configuration C: J 1 =2.1119, J =0.7205 Configuration B with L 1 -optimal design provides 16% improvement w.r.t. open-loop Configuration A with L -optimal design provides 9% improvement w.r.t. open-loop

Example - 4-tanks compartmental system J 1 -optimal closed-loop impulse response - Config. B 1 1 x 1 0.8 0.6 0.4 0.2 0 0 5 10 t 0.2 0.15 x 2 0.8 0.6 0.4 0.2 0 0 5 10 t 0.2 0.15 x 3 0.1 x 4 0.1 0.05 0.05 0 0 5 10 t 0 0 5 10 t

Example - 4-tanks compartmental system J -optimal closed-loop step response - Config. A 1 1 x 1 0.8 0.6 0.4 0.2 x 2 0.8 0.6 0.4 0.2 0 0 5 10 t 1 0.8 0 0 5 10 t 1 0.8 x 3 0.6 0.4 0.2 x 4 0.6 0.4 0.2 0 0 5 10 t 0 0 5 10 t

Example - 4-tanks compartmental system Dual switching design Suppose that the system can be controlled by switching between these two sets of parameters: γ 33 =0.2, γ 34 =0.4, γ 42 =0.5, if γ 33 =0.6, γ 34 =0.2, γ 42 =0.3, if γ(t)=1 γ(t)=2 The system becomes a Dual PMJLS, with γ(t) as a control variable. Design γ(t)=f(x(t),θ(t)), so as to guarantee J 1 < ρ, with minimum ρ.

Example - 4-tanks compartmental system Results of dual switching design Open-loop with γ(t)=1, t: J 1 =2.5164 Open-loop with γ(t)=2, t: J 1 =2.5408 Closed-loop with dual switching: J 1 =2.4965 The dual switching design (after parameter optimization) provides a slight improvement in the L 1 -performance w.r.t. the best open-loop option.

Example - 4-tanks compartmental system Closed-loop impulse response with dual switching 1 1 x 1 0.8 0.6 0.4 0.2 0 0 5 10 t 0.2 0.15 x 2 0.8 0.6 0.4 0.2 0 0 5 10 t 0.2 0.15 x 3 0.1 x 4 0.1 0.05 0.05 0 0 5 10 t 0 0 5 10 t

Conclusions PMJLS are a powerful modeling framework for positive systems with stochastic jumps M-stability assessment and norm optimization can be carried out by means of Linear Programming Efficient tools for the design of state-feedback gains are available, but several problems are still unsolved Weaker notions of stochastic stability (e.g. almost-sure stability) could be considered More effort is required to develop methods for output feedback design The theory of dual switching still needs refinement to achieve less conservative results

Thanks for your attention!