Lecture 6 Verification of Hybrid Systems

Lecture 6 Verification of Hybrid Systems Ufuk Topcu Nok Wongpiromsarn Richard M. Murray AFRL, 25 April 2012 Outline: A hybrid system model Finite-state abstractions and use of model checking Deductive verification and optimization-based construction of certificates Approximate bisimulation functions

Hybrid systems: example gear = 1 ẋ = v v = f 1 (v)u gear = 2 gear = 3 gear = 2 ẋ = v v = f 2 (v)u gear = 3 ẋ = v v = f 3 (v)u gear = 1 gear = 2 Model and tools so far (in the course) help reason about discrete evolution of systems: does there exist a control sequence for which ϕ holds, or do all control sequences lead to executions for which ϕ holds with ϕ = (gear = 1 gear = 3)? Need to modify to capture continuous evolution: Can the car come to a stop from any (reasonable) speed within x meters? How to efficiently accelerate? efficiency,f i f 1 (v) f 2 (v) f 3 (v) v 2

Why to use hybrid system models? Continuous systems with multiple modes Discrete logic controlling continuous systems Continuous systems with hybrid specifications! * & $! *!"!"#$%&'()* +$,-./0&1)(2!&!* #!! "!!!) ( % & " Navigation stack of Alice Mission Planner +,-./012345?:@A @ 9$%*)./( 1$/8 :)/8 '()*&'.0/! '80;8/(&<&:)/8&=>! 6?85-*)./(&=> ' 3,$44.5& 6.,578 % # $ ( '!)!"#$%&'"()&"*$+%)&,-"." /012$3-%*"3-"%)&"40,53-6" 7$%"89$-&":;<"0,&" =)&>52$3-%*""""""""""""""""""""""""""!"!# Traffic Planner Path Planner Vehicle Actuation min T t 0 L(x, u)dt s.t. ẋ = f(x, u) g(x, u) 0 3

A (simple) hybrid system model X Hybrid system: H =(X, L, X 0,I,F,T) with X, continuous state space; I(α 1 ) I(α 3 ) L, finite set of locations (modes); Overall state space X = X L; X 0 X, set of initial states; I : L 2 X, invariant that maps l L to the set of possible continuous states while in location l; F : X 2 Rn, set of vector fields, i.e., ẋ F (l, x); T X X, relation capturing discrete transitions between locations. (l, x) (l,x ) I(α 2 ) L = {α 1, α 2, α 3 } 4

Given: H =(X,L,X 0,I,F,T) Specifications x 0 X 0 Solution at time t with the initial condition : φ(t; x 0 ) With the simple model H, specifying the initial state also specifies the initial mode. Sample temporal properties: Stability: Given equilibrium x e X, for all x 0 X 0 X, φ(t; x 0 ) X, t and φ(t; x 0 ) x e, t Safety: Given X unsafe X, safety property holds if there exists no t unsafe and trajectory with initial condition x 0 X 0, φ(t unsafe ; x 0 ) X unsafe φ(t; x 0 ) X, t [0,t unsafe ] X reach X Reachability: Given, reachability property holds if there exists finite t reach 0 and a trajectory with initial condition, X 0 x 0 X 0 φ(t reach ; x 0 ) X reach and φ(t; x 0 ) X, t [0,t reach ] X reach Eventuality: reachable from every initial condition Combinations of the above, e.g., starting in, reach both X B and X C, but will not be reached before X C is reached while staying safe. X A X B 5

Verification of hybrid systems: Overview Why not directly use model checking? Model checking applied to finite transitions systems exhaustively search for counterexamples... if found, property does not hold. if there is no counterexample in all possible executions, the property is verified. Exhaustive search is not possible over continuous state spaces. Approaches for hybrid system verification: X 1. Construct finite-state approximations and apply model checking preserve the meaning of the properties, i.e., proposition preserving partitions use over - or under -approximations 2. Deductive verification Construct Lyapunov-type certificates Account for the discrete jumps in the construction of the certificate 3. Explicitly construct the set of reachable states Limited classes of temporal properties (e.g., reachability and safety) Not covered in this lecture 6

Hybrid system: Finite-state, under- and over-approximations H =(X,L,X 0,I,F,T) Finite-transition system: TS =(Q,,Q 0 ) Define the map T : Q 2 X q T 1 (q) For discrete state, the corresponding cell in X. X is Under-approximation: TS is an under-approximation of H if the following two statements hold. Given q, q Q with q = q, if q q, then for all x 0 T 1 (q), there exists finite τ > 0 such that φ(τ; x 0 ) T 1 (q ), φ(t; x 0 ) T 1 (q) T 1 (q ), t [0, τ] If q q, then T 1 (q) is positively-invariant. In other words: Every discrete trajectory in an under-approximation TS can be implemented by H. TS simulates H. Over-approximation: TS is an over-approximation of H, if for each discrete transition in TS, there is a possibility to be implemented by H. Possibility induced by the coarseness of the partition. 7

Use of under-approximations Let the following be given. A hybrid system H, a finite-state, under-approximation TS1 for H, Verification Let an LTL specification ϕ be given. Question: H = ϕ? Model check TS1 = ϕ? Words( ϕ) Trace(TS1) Trace(H) Words( ϕ) Trace(TS1) is nonempty Words( ϕ) Trace(H) is nonempty H cannot satisfy the specification. TS1 = φ H = φ Words( ϕ) Trace(TS1) is empty Inconclusive Logic synthesis: If Words(ϕ) Trace(TS1) is nonempty, there exists a trajectory of TS1 which satisfies ϕ and can be implemented by H. Otherwise, inconclusive. 8

Use of over-approximations Hybrid system H and a finite-state, over-approximation TS2 for H. Verification Words(ϕ) Trace(TS2) is nonempty Inconclusive Words( ϕ) Trace(H) Trace(TS2) Words( ϕ) Trace(TS2) is empty Words( ϕ) Trace(H) isempty H satisfies the specification. TS2 = ϕ H = ϕ Logic synthesis: If Words(ϕ) Trace(TS2) is empty, no valid trajectories for TS2 or H. Otherwise, inconclusive. Remarks: Under- and over-approximations give partial results. Potential remedies: Finer approximations Bisimulations 9

Example: verification System models: Continuous vector field: Discrete over-approximation: (small dots: self transitions) Specifications: Both hold for the overapproximation; hence, they hold for the actual system. Example from Temporal logic analysis of gene networks under parametric uncertainty, Batt, Belta, Weiss, Joint special issue of IEEE TAC & Trans on Circuits, 2008. 10

Verification of hybrid systems: Overview Why not directly use model checking? Model checking applied to finite transitions systems exhaustively search for counterexamples... if found, property does not hold. if there is no counterexample in all possible executions, the property is verified. Exhaustive search is not possible over continuous state spaces. Approaches for hybrid system verification: 1. Construct finite-state approximations and apply model checking preserve the meaning of the properties, i.e., proposition preserving partitions use over - or under -approximations 2. Deductive verification Construct Lyapunov-type certificates Account for the discrete jumps in the construction of the certificate 3. Explicitly construct the set of reachable states Limited classes of temporal properties (e.g., reachability and safety) Not covered in this lecture 11

What does deductive verification mean? Example with continuous, nonlinear dynamics: ẋ(t) =f(x(t)) where x(t) R n, f(0) = 0,x=0 is an asymptotically stable equilibrium. Region-of-attraction: R := x : lim φ(t; x) =0 t Question 1 (a system analysis question): Given S R n, is S invariant and S R? the question we want to answer the question we attempt to answer Question 2 (an algebraic question): Does there exist a continuously differentiable function V : R n R V is positive definite, V(0) = 0, Ω := {x : V (x) 1} {x : V f(x) < 0} {0} S Ω? such that Yes to Question 2 Yes to Question 1. 12

Barrier Certificates - Safety Safety property holds if there exists no and trajectory such that: T 0 X X unsafe x = φ(0; x) X initial φ(t ; x) X unsafe φ(t; x) X t [0,T]. X initial Continuous dynamics: ẋ(t) =f(x(t)) Suppose there exists a differentiable function B such that Hybrid dynamics: H =(X,L,X 0,I,F,T ) Suppose there exist differentiable functions (for each mode) such that B l B(x) 0, x X initial B(x) > 0, x X unsafe B x f(x) 0, x X. Then, the safety property holds. B l (x) 0, x I(l) X initial B l (x) > 0, x I(l) X unsafe B l F (x) 0, x I(l) x B l (x ) B l (x) 0, for each jump (l, x) (l,x ) Then, the safety property holds. 13

Barrier Certificates - Eventuality ẋ(t) =f(x(t)) X X target Eventuality property holds if for all x 0 X initial, φ(t ; x 0 ) X target φ(t; x 0 ) X, t [0,T] for some non-negative T. notation: set closure X initial X, X target, X initial are bounded Suppose that f is continuously differentiable and there exists a continuously differentiable function B such that don t leave X before reaching X target leave X \X target in finite time B(x) 0, x X initial B(x) > 0, x X\ X target B x (x) f(x) < 0, x X \X target Then, the eventuality property holds. Straightforward extensions for hybrid dynamics as in safety verification are possible. 14

Composing Barrier Certificates 5 4 3 2 X X A 1 X B x 2 0-1 -2-3 X C If system starts in X A, then both X B and X C are reached in finite time, but X C will not be reached before system reaches X B. -4-5 -5-4 -3-2 -1 0 1 2 3 4 5 B 1 (x) 0 x X A, x 1 B 1 (x) > 0 x X X C, B 1 x (x)f(x, d) ɛ (x, d) (X \X B) D, B 2 (x) 0 x X A, B 2 (x) > 0 x X, Prajna, Ph.D. Thesis, 2005. B 2 x (x)f(x, d) ɛ x (X \X C) D, incorporating disturbances and uncertainties 15

General How procedure to construct to construct the certificates? certificates System properties Algebraic conditions Lyapunov, dissipation inequalities. { Problemdependent! Algebraic conditions Numerical optimization problems Restrict the attention to polynomial vector fields, polynomial certificates,... S-procedure like conditions (for set containment constraints) Sum-of-squares (SOS) relaxations for polynomial nonnegativity Pass to semidefinite programming (SDP) that are equivalent of SOS conditions Solve the resulting (linear or bilinear ) SDPs Construct polynomial certificates { Generally taken care of by software packages. 16

Some preliminaries Semidefinite programming problems Positive semidefinite polynomials and sum-of- squares (SOS) programming Set containment conditions and S-procedure 17

Linear and Bilinear Matrix Inequalities Linear and bilinear matrix inequalities Convex, efficient, generalpurpose solvers exist { Given matrices {F i } N i=0 Sn n, Linear Matrix Inequality (LMI) is a constraint on λ R N of the form: F 0 + N k=1 λ k F k 0 Non-convex, no efficient, generalpurpose solvers { Given matrices {F i } N i=0, {G j} M j=1, and {H k,j} N k=1 M j=1 S n n, a Bilinear Matrix Inequality (BMI) is a constraint on λ R N and γ R M of the form: F 0 + N k=1 λ k F k + M j=1 γ k G j + N k=1 M j=1 λ k γ j H k,j 0 Semidefinite program (?) very roughly speaking, optimization of affine objective subject to LMI or/and BMI constraints 18

Polynomials and Multipoly Toolbox Given α N n, a monomial in n variables is a function m α : R n R defined as m α (x) := x α 1 1 xα 2 2 xα n n. The degree of a monomial is defined as deg m α := n i=1 α i. Polynomial: Finite linear combination of monomials. : R n R p := c α m α = c α x α where A N n is a finite set and c α R α A. α A α A (available through workshop web page) { } Multipoly is a Matlab toolbox for the creation and manipulation of polynomials of one or more variables. ultipoly Toolbox Example: pvar x1 x2 p = 2*x1^4 + 2*x1^3*x2 - x1^2*x2^2 + 5*x2^4 q = x1^2 p*q = 2*x1^6 + 2*x1^5*x2 - x1^4*x2^2 + 5*x1^2*x2^4 jacobian(p, [x1;x2]) = [ 8*x1^3 + 6*x1^2*x2-2*x1*x2^2, 2*x1^3-2*x1^2*x2 + 20*x2^3 ] A A 19

Positive semidefinite polynomials Positive Semidefinite Polynomials R[x 1,...,x n ] or R[x] denotes the set of polynomials (with real coefficients) in the variables {x 1,...,x n }. p R [x] is positive semi-definite (PSD) if p(x) 0 x. The set of PSD polynomials in n variables {x 1,...,x n } will be denoted P [x 1,...,x n ] or P [x]. Testing if p P [x] is NP-hard when the polynomial degree is at least four. For a general class of functions, verifying global non-negativity is recursively undecidable. How about a quadratic polynomial? Reference: Parrilo, P., Structured Semidefinite Programs and Semialgebraic Geometry Methods in Robustness and Optimization, Ph.D. thesis, California Institute of Technology, 2000. (Chapter 4 of this thesis and the reference contained therein summarize the computational issues associated with verifying global non-negativity of functions.) 20

Sum-of-Squares Polynomials m of Squares Polynomials p is a sum of squares (SOS) if there exist polynomials {f i } N i=1 such p is that a sump of = squares N (SOS) if there exist polynomials {f i } N i=1 such that p = i=1 f i 2. The set of SOS polynomials N i=1 f i 2. in n variables {x 1,...,x n } will be The denoted set of SOS Σ [x 1 polynomials,...,x n ] or in Σ n [x]. variables {x 1,...,x n } will If p is a SOS then p is PSD. be denoted Σ [x 1,...,x n ] or Σ [x]. If p is a SOS then p is PSD. The Motzkin polynomial, p = x 2 y 4 + x 4 y 2 +1 3x 2 y 2, is PSD but not SOS. p Hilbert (1888) showed that P [x] =Σ [x] only for a) n =1, b) d =2, and c) d =4, n =2. p(x) =z(x) T Qz(x) For every polynomial matrix Q such that of degree 2d, there exists a symmetric with p is a SOS iff there exists Q 0 such that p = z T Qz. z(x) := 1,x 1,...,x n,x 2 1,x 1 x 2,...,x 2 n,...,x d n T Reference: Choi, M., Lam, T., and Reznick, B., Sums of Squares of Real Polynomials, Proceedings of Symposia in Pure Mathematics, Vol. 58, No. 2, 1995, pp. 103 126. p is SOS if and only if there exists Q 0 s.t. p(x) =z(x) T Qz(x) Given p, can be verified as SDP. 21

SOS SOS Example Example SOS example All All possible possible Gram Gram matrix matrix representations representations of of p(x) =2x 4 1 +2x 3 1x 2 x 2 1x 2 2 +5x 4 2 p =2x =2x 4 1 4 +2x 1 +2x 3 1x 3 2 1x 2 x 2 1x 2 1x 2 2 +5x 2 +5x 4 2 4 2 are are given given by by z T (Q (Q + λn) λn) z where: where: x 2 1 2 2 1 0.5 1 2 1 0.5 z = x 1 x 2 1 2,Q= 1 0 0,Q=,N= 1 0 0,N= x 2 2 2 0.5 0 5 0.5 0 5 p(x) =z(x) T Qz(x) 0=z(x) T Nz(x) (x 1 x 2 ) (x 1 x 2 )=x 2 1 x 2 2 0 0 0.5 0 1 0 0.5 0 0.5 0 01 0 0.5 0 0 p is is SOS SOS iff iff Q + λn λn 0 for for some some λ R. R. LMI 22

GeneralSOS programming Programming SOS Programming: Given c R m and polynomials {f j,k } N s j=1 m k=0, solve: min α R ct α m subject to: f 1,0 (x)+f 1,1 (x)α 1 + + f 1,m (x)α m Σ [x]. f Ns,0(x)+f Ns,1(x)α 1 + + f Ns,m(x)α m Σ [x] There is freely available software (e.g. SOSTOOLS, YALMIP, SOSOPT) that: 1. Converts the SOS program to an SDP 2. Solves the SDP with available SDP codes (e.g. Sedumi) 3. Converts the SDP results back into polynomial solutions 23

Set containment conditions Given polynomials g 1 and g 2, define sets S 1 and S 2 : S 1 := {x R n : g 1 (x) 0} S 2 := {x R n : g 2 (x) 0} Is S 2 S 1? Polynomial S-procedure λ Σ[x] s.t. g 1 (x)+λ(x)g 2 (x) Σ[x] λ positive semidefinite polynomial s.t. g 1 (x)+λ(x)g 2 (x) 0 x {x : g 2 (x) 0} { x : g 1 (x) 0} Example: B(x) 0, x X initial Suppose X initial = {x : g(x) 0} for some g Sufficient condition: There exists positive semidefinite function s such that B(x)+s(x)g(x) = B(x) s(x)( g(x)) 0, x R n 24

Global Stability Theorem Global stability theorem Theorem: Let l 1,l 2 R [x] satisfy l i (0) = 0 and l i (x) > 0 x = 0 for i =1, 2. If there exists V R [x] such that: V (0) = 0 V l 1 Σ [x] V f l 2 Σ [x] Then R 0 = R n. Reference: Vidyasagar, M., Nonlinear Systems Analysis, SIAM, 2002. (Refer to Section 5.3 for theorems on Lyapunov s direct method.) V is positive definite, radially unbounded V is decreasing along the vector field 25

Global Stability Example with sosopt Global stability examples with sosopt % Code from Parrilo1_GlobalStabilityWithVec.m % Create vector field for dynamics pvar x1 x2; x = [x1;x2]; x1dot = -x1-2*x2^2; x2dot = -x2 - x1*x2-2*x2^3; xdot = [x1dot; x2dot]; % Use sosopt to find a Lyapunov function % that proves x = 0 is GAS % Define decision variable for quadratic % Lyapunov function zv = monomials(x,2); V = polydecvar( c,zv, vec ); % Constraint 1 : V(x) - L1 \in SOS L1 = 1e-6 * ( x1^2 + x2^2 ); sosconstr{1} = V - L1; % Constraint 2: -Vdot - L2 \in SOS L2 = 1e-6 * ( x1^2 + x2^2 ); Vdot = jacobian(v,x)*xdot; sosconstr{2} = -Vdot - L2; x2 % Solve with feasibility problem [info,dopt,sossol] = sosopt(sosconstr,x); Vsol = subs(v,dopt) Vsol = 0.30089*x1^2 + 1.8228e-017*x1*x2 + 0.6018*x2^2 4 3 2 1 0!1!2!3 5 5 2 2 1 5 5 1 0.5 2!4!4!2 0 2 4 x1 0.1 2 0.5 1 5 5 2 5 26

Approximate bisimulation relations & bisimulation functions Two systems with x i R n i,x i (0) I i R n i,u i (t) U i R m i,y i R p Φ 1 : ẋ1 (t) =f 1 (x 1 (t),u 1 (t)) y 1 (t) =g 1 (x 1 (t)) Φ 2 : ẋ2 (t) =f 2 (x 2 (t),u 2 (t)) y 2 (t) =g 2 (x 2 (t)) A relation R δ R m 1 R n 2 is a δ-approximate bisimulation relation between Φ 1 and Φ 2 if for all (x 1,x 2 ) R δ : g 1 (x 1 ) g 2 (x 2 ) δ; T >0 and u 1 ( ), u 2 ( ) s.t. (φ 1 (t; x 1 ) φ 2 (t; x 2 )) R δ t [0,T]; T >0 and u 2 ( ), u 1 ( ) s.t. (φ 1 (t; x 1 ) φ 2 (t; x 2 )) R δ t [0,T]. If start in relation, stay in relation. Observations are close. A function V : R n 1 R n 2 R + {+ } is a bisimulation function between Φ 1 and Φ 2 if for all δ 0: R δ = {(x 1,x 2 ) R n 1 R n 2 : V (x 1,x 2 ) δ} sublevel sets of V induce a relation is a closed set and a δ-approximate bisimulation relation between Φ 1 and Φ 2. 27

Approximate bisimulation relations & bisimulation functions Two systems with x i R n i,x i (0) I i R n i,u i (t) U i R m i,y i R p Φ 1 : ẋ1 (t) =f 1 (x 1 (t),u 1 (t)) y 1 (t) =g 1 (x 1 (t)) Φ 2 : ẋ2 (t) =f 2 (x 2 (t),u 2 (t)) y 2 (t) =g 2 (x 2 (t)) Let W : R n 1 R n 2 R + be a continuously differentiable function. If for all (x 1,x 2 ) R n 1 R n 2, W (x 1,x 2 ) g 1 (x 1 ) g 2 (x 2 ) 2 W f 1 (x 1,u 1 ) W f(x 2,u 2 ) 0, (x 1,x 2 ) R n 1 R n 2,u 1 R m 1,u 2 R m 2 x 1 x 2 then V := W is a bisimulation function between Φ 1 and Φ 2. guarantees that no matter what u 1 and u 2 do, the time derivative of W stays non-positive 28

Approximate bisimulations + safety 0 0 5 Reach [0,T ] (Φ 1 ) 5 Reach [0,T ] (Φ 2 ) 10 Unsafe 10 Unsafe 15 2 4 6 8 10 12 14 16 18 20 22 15 2 4 6 8 10 12 14 16 18 20 22 Φ 1 s.t. x 1 R 10 Φ 2 s.t. x 2 R 4 Φ 3 s.t. x 3 R 6 0 5 Reach [0,T ] (Φ 3 ) Φ 1 is 1.90-approximate bisimilar to Φ 2 Φ 1 is 0.76-approximate bisimilar to Φ 3 10 Unsafe Figures from Girard & Pappas, Automatica, 2007. 15 2 4 6 8 10 12 14 16 18 20 22 29