Cyber-Physical Systems Modeling and Simulation of Hybrid Systems

Cyber-Physical Systems Modeling and Simulation of Hybrid Systems Matthias Althoff TU München 05. June 2015 Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 1 / 28

Overview Overview Hybrid Systems Modeling as hybrid automata Other modeling formalisms Properties of hybrid systems Numerical simulation of hybrid automata Stability analysis of hybrid systems (next lecture) Reachability analysis of hybrid systems (next lecture) Hybrid systems are the most general class of systems considered in this course. Timed automata can already be considered as hybrid systems with a simple continuous dynamics (ċ i = 1). Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 2 / 28

Modeling as Hybrid Automata Why combine discrete and continuous dynamics? In many cyber-physical systems, the continuous dynamics and the discrete dynamics cannot be designed separately (see first lecture): automated driving source: Carnegie Mellon University human-robot collaboration source: Rethink Robotics smart grids source: Siemens automated farming surgical robots air traffic control source: Kesmac source: davinci source: NASA Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 3 / 28

Hybrid Automata Modeling as Hybrid Automata Hybrid automata describe the dynamics of systems that can be described by a finite set of discrete states z i and continuous state variables x i R. Starting from an initial state z(t 0 ), initial continuous values x i (t 0 ), a continuous input trajectory u c (t), and a timed input sequence ū = ( (u(t 0 ), t 0 ), (u(t 1 ), t 1 ), (u(t 2 ), t 2 ),... ) a finite state automaton creates a continuous output trajectory y c (t) and a timed output sequence ȳ = ( (y( t 0 ), t 0 ), (y( t 1 ), t 1 ), (y( t 2 ), t 2 ),... ), where it is not required that the times t i and t i are synchronized. ū u c(t) Hybrid automaton ȳ y c (t) Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 4 / 28

Modeling as Hybrid Automata Syntax of Hybrid Automata Definition A hybrid automaton HA is a tuple (ordered set): HA = (Z, X, U, Y, U c, Y c, T, inv, g, h, f, z 0, x 0 ), where z 0 is the discrete initial state, x 0 is the continuous initial state Z = {z 1,..., z n } X R n U = {ũ 1,..., ũ p } Y = {ỹ 1,..., ỹ q } U c R m Y c R o T Z U Z Y inv : Z P(X ) g : T P(X ) h : T X X f : Z X U c R n set of discrete states continuous state space set of input symbols (input alphabet) set of output symbols (output alphabet) continuous input space continuous output space set of transitions invariant function guard function jump function flow function Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 5 / 28

Modeling as Hybrid Automata Semantics of Hybrid Automata Our definition of a hybrid automaton has the following semantics: The hybrid automaton starts at the discrete state z 0 and the continuous state x 0. The continuous state evolves according to the flow function that is assigned to each location z i : ẋ = f (z i, x, u c ). As soon as the continuous state x is within a guard set g(z, u, z, y) of a transition (z, u) (z, y), the corresponding transition is activated. As soon as the input event u of an activated transition occurs, the transition is taken and the output event y is generated. A transition is enforced if the continuous state would leave the invariant inv(z i ). After a transition is taken, the jump function resets the continuous state: x = h ( (z, u, z, y), x ) Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 6 / 28

Modeling as Hybrid Automata Animation of Hybrid Automata trajectory invariant initial continuous set x 1 guards z x 1 z 2 2 Continuous evolution Start at z 0 and x 0 x(t) is the solution of ẋ(t) = f (z(t), x(t), u c (t)) Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 7 / 28

Modeling as Hybrid Automata Animation of Hybrid Automata trajectory invariant initial continuous set x 1 guards z x 1 z 2 2 Activation of discrete transition Transition (z, u, z, y) is activated when x(t) g(z, u, z, y) (z: before transition, z : after transition) Transition is taken as soon as event u occurs Transition is enforced when x(t) leaves inv(z) Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 7 / 28

Modeling as Hybrid Automata Animation of Hybrid Automata invariant initial continuous set trajectory jump x 1 guards z x 1 z 2 2 Discrete transition and jump of continuous state Location changes from z to z Output event y is generated Continuous state may jump: x = h ( (z, u, z, y), x ) (x : continuous state after jump) Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 7 / 28

Modeling as Hybrid Automata Animation of Hybrid Automata invariant initial continuous set trajectory jump x 1 guards z x 1 z 2 2... and so on... Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 7 / 28

Modeling as Hybrid Automata Hybrid Automaton of a Bouncing Ball Given is a ball with dynamics s = g, where s is the vertical position and g is the gravity constant. After impact with the ground at s = 0, the velocity changes to v = αv (v = ṡ) with α [0, 1]. s 0 v 0 g Z = {z 1 } X = R + R (ball above ground) U = Y = {ǫ} U c = Y c = {} T = {(z 1, ǫ, z 1, ǫ)} inv(z 1 ) = {[x 1, x 2 ] T x 1 R + 0, x 2 R} g ( (z 1, ǫ, z 1, ǫ) ) = {[x 1, x 2 T x 1 = 0, x 2 R 0 } h ( (z 1, ǫ, z 1, ǫ), x ) [ ] x1 = [ αx ] 2 x2 f(z 1, x) = g Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 8 / 28

Modeling as Hybrid Automata Graphical Representation of the Bouncing Ball A typical representation of hybrid automata is as follows: Discrete states are represented by circles (or similar shapes). Transitions are illustrated by arrows to which input and output events, guards, and jump functions are attached. The continuous dynamics is written within the discrete states above a dashed line. The invariant is placed underneath the dashed line. differential equations ẋ 1 = x 2 ẋ 2 = g x 1 = 0 x 2 0 guard invariant x 1 0 x 2 := αx 2 jump function Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 9 / 28

Modeling as Hybrid Automata Trajectory of the Bouncing Ball The trajectories of the bouncing ball are plotted for s 0 = 30 [m] (quite high, but then we can reasonably plot the result together with velocity), v 0 = 0 [m/s], and α = 0.8 [-]. 30 20 s(t) s(t), v(t) 10 0 10 20 v(t) 30 0 2 4 6 8 10 12 14 16 18 20 t Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 10 / 28

Other Modeling Formalisms Hybrid Statecharts When a hybrid automata has many locations, it is useful to group them using statecharts. Guards: Are modeled as conditions of discrete transitions. Flow function: Is specified after the keyword throughout (MATLAB/Stateflow: during/du) within a state. Jump function: Is specified after the keyword exit within a state or attached to a discrete transition. Invariant: Most work does not specify invariants for statecharts. Instead, an urgent semantics is assumed, i.e. a transition is taken as soon as a state is in a guard. Why does one not require invariants in this case? Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 11 / 28

Other Modeling Formalisms Hybrid Statecharts: Electric Motor (I) We model an electric motor with several operation modes. The torque T = k m i of our motor is proportional to the applied current i. We also model friction as T f = k f ω and denote the disturbance torque by T d so that the overall dynamics of the angular velocity ω for the rotational inertia J is J ω = T = k m i k f ω + T d. To control the speed, we use a simple P-controller i = K P (ω d ω) so that we obtain ω = k m J K P(ω d ω) k f ω + T d. The motor has the following modes: The motor is switched off. The motor is fully accelerating with current i max until the speed ω d is reached or when the speed drops below 0.8ω d. The motor is controlled to keep the speed ω d. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 12 / 28

Other Modeling Formalisms Hybrid Statecharts: Electric Motor (II) power on power off on acceleration ω ω d const speed during: ω = f 1(ω, T d ) during: ω = f 2(ω, T d ) during: ω = f 3(ω, T d ) off ω < 0.8ω d f 1 (ω, T d ) = k f ω + T d, f 2 (ω, T d ) = k m J i max k f ω + T d, f 3 (ω, T d ) = k m J K P(ω d ω) k f ω + T d. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 13 / 28

Other Modeling Formalisms Hybrid Statecharts: Bouncing Ball in MATLAB/Stateflow In a similar way, one can model hybrid statecharts in MATLAB/Stateflow. MATLAB also assume urgent semantics, i.e. a transition is taken as soon as a state is in a guard. The bouncing ball example in MATLAB/Stateflow: Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 14 / 28

Other Modeling Formalisms Interaction of Discrete and Continuous Components (I) Many engineering tools realize hybrid systems by combining discrete and continuous components. No explicit modeling formalism: Discrete states, guards, etc. are implicitly described by the interaction of continuous and discrete components. Example: Bouncing ball modeled in MATLAB/Simulink Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 15 / 28

Other Modeling Formalisms Interaction of Discrete and Continuous Components (II) Advantages Intuitive modeling. Components can be easily exchanged, which might cause changing the discrete transition structure of many locations in a hybrid automata. Disadvantages The modeling formalism is not suited for formal analysis. The modeling formalism is not formally defined. What happens when several discrete components switch at the same time? Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 16 / 28

Properties of Hybrid Automata Deadlock and Livelock Due to bad design, the undesired event of deadlock and livelock can occur. Deadlock A deadlock occurs when the invariant continuous state leaves the invariant x 2 and is not in any guard set. x 1 guard trajectory deadlock Livelock A livelock occurs when the system switches infinitely often between discrete states and no time passes in between discrete transitions. Example: A continuous state g ( (z 1, ǫ, z 2, ǫ) ) g ( (z 2, ǫ, z 1, ǫ) ) jumps in between guard sets x 2 that cause transitions in x between each other. 1 inv(z 1) inv(z 2) Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 17 / 28

Nondeterminism Properties of Hybrid Automata Since finite state automata are a special case of hybrid automata, it is obvious that hybrid automata can be nondeterministic. The sources of nondeterminism are manifold: Guard regions can overlap so that several goal locations are possible. The jump function can be nondeterministic. When the guard set is full-dimensional (see figure below), the switching time is nondeterministic. The differential equations have uncertain continuous inputs. One can define hybrid automata with nondeterministic initial states. g ( (z g ( (z 1, ǫ, z 2, ǫ) ) 1, ǫ, z 2, ǫ) ) x 2 x 1 inv(z 1) deterministic inv(z 1) non-deterministic Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 18 / 28

Zeno Behavior (I) Properties of Hybrid Automata Zeno behavior occurs when the duration δ i between the i th and the (i + 1) th transition decreases and i=0 δ i is finite. With other words, an infinite number of transitions occurs in finite time. Example: Bouncing Ball Let us introduce the velocity v i and the time t i at the i th transition: v i+1 = αv i t i+1 = t i + 2α g v i ( α α i+1 ) v i = α i v 0 t i = t 0 + 2v 0 g v zeno = lim i v i = 0 (we use n 1 k=0 axk = a 1 x n 1 x (x 1) ) 1 α t zeno = lim i t i = t 0 + 2v 0 g ( α ) 1 α Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 19 / 28

Zeno Behavior (II) Properties of Hybrid Automata For the values v 0 = 1 [m/s] (here: velocity at initial contact), α = 0.8 we have t zeno = 2 4 0.82 [s] g The ball dynamics cannot proceed beyond 0.82 [s]. In reality, the elasticity of the ball causes the ball not take off after a certain time. Zeno of Elea Greek philosopher who is famous for his paradoxes, such as the one of Achilles and the tortoise: A tortoise wants to race against Achilles and he gives it a head start. After both start running, the tortoise has already moved to s 1 when Achilles arrives at its initial position s 0. Then Achilles runs to s 1 when the tortoise is at s 2. Zeno claims that by this argumentation, Achilles can never overtake the tortoise. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 20 / 28

Finite Escape Time Properties of Hybrid Automata One speaks of finite escape time, when x in finite time. Finite escape time is also possible for purely continuous systems when they are nonlinear: ẋ = 1 + x 2 (t), x 0 = 0 The solution of the differential equation is x(t) = tan(t), which has an explosion time at t = π 2. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 21 / 28

Numerical Simulation of Hybrid Automata Numerical Simulation of Hybrid Automata As for nonlinear continuous systems, for most hybrid systems modeling real world problems, there exists no analytical solution. Steps in hybrid system simulation 1 Simulation of the continuous dynamics within the current location (see lecture Modeling and Simulation of Continuous Systems ) as long as the state is in the invariant; 2 Detection whether the current state is within a guard set and whether it is activated by the required input event; 3 Update of the discrete state once the transition is taken and generate the output event; 4 Update of the continuous state according to the jump function; 5 Continue with step 1. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 22 / 28

Numerical Simulation of Hybrid Automata Guard detection Step 1 (continuous evolution) has been previously discussed and is e.g. performed via Runge-Kutta methods. Step 3 and 4 (discrete and continuous update) are trivial. We need to focus on step 2 (guard detection). For simplicity we only consider deterministic guards, i.e. guards that can only be hit at one point in time. Those guards are usually only activated by the state: g ( (z, ǫ, z, ǫ) ). Why? Reminder: g ( (z 1, ǫ, z 2, ǫ) ) g ( (z 1, ǫ, z 2, ǫ) ) x 2 x 1 inv(z 1) deterministic inv(z 1) non-deterministic Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 23 / 28

Numerical Simulation of Hybrid Automata Modeling of Guards We model the guard by a level set function l(x), which allows arbitrary shapes: Examples: g ( (z, ǫ, z, ǫ) ) = {x l(x) = 0}. hyperplane: l(x) = n T x d, where n R n is the normal vector and d R is the distance from the origin to the hyperplane. x 2 d x 1 hyperplane hypersphere: l(x) = x c 2 r, where c R n is the center and r R is the radius of the hypersphere. r n T hypersphere x 2 x 1 c Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 24 / 28

Numerical Simulation of Hybrid Automata Guard Detection without Hitting Time Detection A guard has been crossed when the level set function l(x) changes its sign: trajectory with exact hitting time detection trajectory in z 2 without hitting time detection z 2 x 2 x 1 z 1 l(x) < 0 trajectory in z 1 guard: l(x) = 0 x(t n) x(t n+1) l(x) > 0 A simple method is to perform a discrete transition as soon as a sign change of l(x) is detected, without determining the exact switching time. This is computationally cheap, but creates larger errors. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 25 / 28

Numerical Simulation of Hybrid Automata Guard Detection with Hitting Time Detection More accurate results are obtained when the solver iteratively searches for the exact hitting time until the value of l(x) is in a ǫ-region: l(x) 2 ǫ. trajectory with exact hitting time detection z 2 4 1 x 2 x 1 z 1 2 l(x) < 0 x(t n) x(t n+1) trajectory in z 1 3 guard: l(x) = 0 l(x) > 0 An iterative method for hitting time detection is presented in the exercise. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 26 / 28

Further Reading Numerical Simulation of Hybrid Automata A. van der Schaft and H. Schumacher: An Introduction to Hybrid Dynamical Systems, Springer, 2000. R. Alur, C. Coucoubetis, N. Halbwachs, T.A. Henzinger, P.H. Ho, X. Nicolin, A. Olivero, J. Sifakis, S. Yovine: The Algorithmic Analysis of Hybrid Systems, Theoretical Computer Science, 1995, 138, pages 3-34. Y. Kesten and A. Pnueli: Timed and Hybrid Statecharts and their textual representation, Formal Techniques in Real-Time and Fault-Tolerant Systems, LNCS 571, 1991, pages 591-620. M. Otter, H. Elmqvist, and Sven Erik Mattsson: Hybrid Modeling in Modelica based on the Synchronous Data Flow Principle, Proc. of Computer Aided Control System Design, 1999. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 27 / 28

Conclusions Numerical Simulation of Hybrid Automata In many cyber-physical systems, the continuous dynamics and the discrete dynamics cannot be designed separately. Hybrid automata are an extension of finite state automata by continuous dynamics. There exists a large number of alternative modeling formalisms: Hybrid statecharts, hybrid Petri nets, hybrid bond graphs, etc. Hybrid systems can exhibit a variety of phenomena: Deadlock Livelock Nondeterminism Zeno behavior Finite escape time The main difficulty in extending numerical solvers for continuous systems is guard detection. Matthias Althoff Modeling and Simulation of Hybrid Systems 05. June 2015 28 / 28

Cyber-Physical Systems Analysis of Hybrid Systems Matthias Althoff TU München 12. June 2015 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 1 / 39

Overview Overview Hybrid Systems Stability analysis of hybrid systems: Common Lyapunov function Multiple Lyapunov function Reachability analysis of hybrid systems Applications Matthias Althoff Analysis of Hybrid Systems 12. June 2015 2 / 39

Stability Analysis of Hybrid Systems Motivating Example (I) Warning Even if a hybrid system is Lyapunov stable in all locations, the hybrid system is not necessarily stable! Example: Hybrid automaton with two locations Z = {z 1, z 2 }, two continuous state variables X = {x 1, x 2 }, and no inputs and outputs: T = {(z 1, ǫ, z 2, ǫ), (z 2, ǫ, z 1, ǫ)} inv(z 1 ) = {[x 1, x 2 ] T x 1 x 2 0, x 1 R, x 2 R} inv(z 2 ) = {[x 1, x 2 ] T x 1 x 2 0, x 1 R, x 2 R} g ( (z 1, ǫ, z 2, ǫ) ) = g ( (z 2, ǫ, z 1, ǫ) ) = {[x 1, x 2 ] T x 1 x 2 = 0, x 1 R, x 2 R} h ( (z 1, ǫ, z 2, ǫ), x ) = h ( (z 2, ǫ, z 1, ǫ), x ) [ ] = x 1 4 f(z 1, x) = x [ 1 1 ] 1 1 f(z 2, x) = x 4 1 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 3 / 39

Stability Analysis of Hybrid Systems Motivating Example (II) The phase portraits of each subsystem are as follows: location z 1 : x2 3 2 1 0 1 2 3 location z 2 : x2 1.5 1 0.5 0 0.5 1 1.5 3 2 1 0 x 1 1 2 3 1 0 1 x 1 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 4 / 39

Stability Analysis of Hybrid Systems Motivating Example (III) It can be seen from the phase portrait, that the system is unstable (left figure). When exchanging the flow functions, the system is stabilized (right figure). Trajectory of original switching sequence: Trajectory of modified switching sequence: x2 100 80 60 40 20 0 20 40 90 80 70 60 50 40 30 20 10 0 10 x 1 x2 1.2 1 0.8 0.6 0.4 0.2 0 0.2 1 0.8 0.6 0.4 0.2 0 0.2 0.4 x 1 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 5 / 39

Common Lyapunov Function Common Lyapunov Function We first address the problem that we have a system of which the modes can be arbitrarily switched. Arbitrary switching can be modeled by defining i : g(t i ) = R n, where T i is the i th transition. External events are used to perform the switching. Common Lyapunov function is sufficient (proof omitted) If the continuous systems of all locations share a common Lyapunov function, the hybrid dynamics is stable. Common Lyapunov function is necessary (proof omitted) If a hybrid system is stable for arbitrary sequences of locations, all locations share a common Lyapunov function. A common Lyapunov function is necessary and sufficient. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 6 / 39

Common Lyapunov Function Lyapunov Function for Linear Systems Given is a linear system ẋ(t) = Ax(t). (1) Lyapunov function for LTI systems The Lyapunov function V (x) = x T Px, P > 0 proves that an LTI system is stable if PA + A T P < 0 Proof: Using (AB) T = B T A T, we have that V (x) = x T Pẋ + ẋ T Px = x T PAx + x T A T Px = x T (PA + A T P)x so that PA + A T P < 0 when the system is stable. Lyapunov function is necessary (no proof) One can show that if (1) is stable there has to exist a P such that PA + A T P < 0. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 7 / 39

Common Lyapunov Function Common Lyapunov Function: Switched Linear Systems For switched linear systems ẋ(t) = A (i) x(t) where i refers to the i th location, it is natural to use the quadratic Lyapunov function V (x) = x T Px, P > 0 so that PA (i) + (A (i) ) T P < 0 when the i th location has a stable dynamics. This problem can be written as linear matrix inequalities for which powerful solvers exist: P > 0 i : A (i) P + PA (i) < 0 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 8 / 39

Common Lyapunov Function Common Lyapunov Function: Infeasibility Test For switched linear systems there exists an infeasibility test for quadratic Lyapunov functions: Infeasibility Test (no proof) If there exist M positive definite matrices R (i) > 0 (M: number of locations) such that M R (i) (A (i) ) T + A (i) R (i) > 0 i=1 then there is no P > 0 such that i {1,..., M} : (A (i) ) T P + PA (i) < 0 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 9 / 39

Common Lyapunov Function Example for the Infeasibility Test Does stability of a switched linear system imply existence of a common quadratic Lyapunov function? No, the system A (1) = [ ] 1 1, A (2) = 1 1 [ 1 ] 10 0.1 1 is stable for arbitrary switching, but does not have a common quadratic Lyapunov function since [ ] [ ] R (1) 0.2996 0.7048 =, R (2) 0.2123 0.5532 = 0.7048 2.4704 0.5532 1.9719 satisfy the infeasibility condition. However, there is a common piecewise quadratic Lyapunov function. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 10 / 39

Multiple Lyapunov Function Multiple Lyapunov Function It is often easier to use a different Lyapunov function V (z i, x) for each location z i. Lyapunov s stability theorem for hybrid systems The origin is a stable equilibrium of a hybrid automata if for all z i Z and x D 1 V (z i, 0) = 0, x D \ {0} : V (z i, x) > 0 2 V (zi, x) 0, x D 3 For all discrete transition times t i we have that for t i > t j and z(t i ) = z(t j ) that V (z(t i ), x(t i )) < V (z(t j ), x(t j )). One of the difficulties is that one has to know the discrete sequences in advance and that they strongly depend on the initial state. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 11 / 39

Multiple Lyapunov Function Possible Evolution of Lyapunov Function Values V (z, t) V (z 1, t) active inactive V (z 2, t) t 0 t 1 t 2 t 3 t 4 t Matthias Althoff Analysis of Hybrid Systems 12. June 2015 12 / 39

Reachability Analysis of Hybrid Systems Reachability Analysis initial set jump exact reachable set x 1 x 2 possible trajectory steady state Informal Definition A reachable set is the set of states that can be reached by a dynamical system in finite or infinite time for a set of initial states, uncertain inputs, and uncertain parameters. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 13 / 39

Verification Task Reachability Analysis of Hybrid Systems overapproximative reachable set exact reachable set initial set x 1 x 2 invariant set unsafe set Verification Task Check if a set of unsafe states is never reached. Exact reachable set only for special classes computable overapproximation computed for consecutive time intervals. Overapproximation might lead to spurious counterexamples. Simulation cannot prove correctness. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 14 / 39

Reachability Analysis of Hybrid Systems Linear Systems Linear Systems: Overview of Reachable Set Computation ẋ(t) = Ax(t) + u(t), A R n n, x(t) R n, x(0) R(0), u(t) u c U 1 Compute reachable set H(r) = e Ar R(0) r t=0 ea(r t) dt u c at time r neglecting the uncertain input (C D := {c + d c C, d D}). 2 Obtain convex hull of initial set R(0) and H(r). 3 Enlarge reachable set to account for (1) uncertain inputs, (2) curvature of trajectories. 4 Continue with further time intervals [kr, (k + 1)r], k N. Known algorithm, similar to work of A. Girard at HSCC 05. R(0) H(r) convex hull of R(0), H(r) ➀ ➁ ➂ R([0, r]) enlargement Matthias Althoff Analysis of Hybrid Systems 12. June 2015 15 / 39

Reachability Analysis of Hybrid Systems Nonlinear Systems Nonlinear Reachability Analysis: Overall Algorithm initial set R(0), input set U, time step k = 1 linearize system compute reachable set R lin without linearization error k := k + 1 obtain set of linearization errors L based on R lin and L (L: set of admissible linearization errors) L L? yes no compute reachable set R err due to L enlarge L R = R lin R err Matthias Althoff Analysis of Hybrid Systems 12. June 2015 16 / 39

Reachability Analysis of Hybrid Systems Overall Algorithm: Animation Nonlinear Systems R(0) linearize system Matthias Althoff Analysis of Hybrid Systems 12. June 2015 17 / 39

Reachability Analysis of Hybrid Systems Overall Algorithm: Animation Nonlinear Systems R lin ([0, r]) compute reachable set R lin without linearization error Matthias Althoff Analysis of Hybrid Systems 12. June 2015 17 / 39

Reachability Analysis of Hybrid Systems Nonlinear Systems Overall Algorithm: Animation R lin ([0, r]) R err ([0, r]) R err: reachable set due to L obtain set of linearization errors L based on R lin ([0, r]) R err([0, r]) Matthias Althoff Analysis of Hybrid Systems 12. June 2015 17 / 39

Reachability Analysis of Hybrid Systems Overall Algorithm: Animation Nonlinear Systems R([0, r]) = R lin ([0, r]) R err ([0, r]) L L? Matthias Althoff Analysis of Hybrid Systems 12. June 2015 17 / 39

Reachability Analysis of Hybrid Systems Overall Algorithm: Animation Nonlinear Systems R([r, 2r]) reachable set of next time interval Matthias Althoff Analysis of Hybrid Systems 12. June 2015 17 / 39

Reachability Analysis of Hybrid Systems Overall Algorithm: Animation Nonlinear Systems possible trajectories R([0, t f ]) reachable set of the complete time horizon t f Matthias Althoff Analysis of Hybrid Systems 12. June 2015 17 / 39

Reachability Analysis of Hybrid Systems Nonlinear Systems Scalability of the Linearization Approach 6 x 1 u 5 initial set x n 1. (more tanks) x 6 4 3 possible trajectories x n Water tank system. 2 1 2 3 4 x 1 Projected reachable set (n = 6). Complexity with respect to the number of continuous state variables n: O(n 3 ). Dimension n 5 10 20 50 100 CPU-time [sec] 1.19 1.73 3.11 11.59 35.78 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 18 / 39

Reachability Analysis of Hybrid Systems Hybrid Systems Reachability Analysis of Hybrid Systems Hybrid systems additionally require intersections of guard sets: guard R g guard R g R(t η) R([t k, t k+1 ]) x 2 x 1 R(0) (a) Classical approach. R([t k, t k+1 ]) x 2 x 1 R(0) (b) New approach. t η: last point in time before intersecting the hyperplane. R g : Overapproximation of the guard set intersection. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 19 / 39

Reachability Analysis of Hybrid Systems Hybrid Systems Scalability of the Mapping-Based Approach T m engine u dynamics k s k 1 k 2 k θ J m 2α J J1 J 2 J l θ gear Θ s Θref 80 60 40 20 guard set sample traj. 0 R(0) 0.1 0 0.1 0.2 Θ 1 Θ 2 Θ θ Θ m Θ l Powertrain with backlash. Θ s Θ 1 Projected reachable set (n = 101). Complexity with respect to the number of continuous state variables n: O(n 5 ). Dimension n 11 21 31 41 51 101 CPU time [sec] 8.122 14.31 23.72 31.83 53.74 1550 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 20 / 39

Reachability Analysis of Hybrid Systems Comparison With SpaceEx Hybrid Systems SpaceEx: state of the art tool for reachability analysis of hybrid systems. Uses geometric guard intersection. Example sensitive to overapproximation comparison for initial set with 5% of initial size and n = 7. Tm 80 60 40 20 0 R 0.05 (0) SpaceEx mapping approach guard 20 0.05 0 0.05 0.1 Θ s Θ 1 Θref 80 60 40 20 R 0.05 (0) Computational times: 10023 s (new approach: 0.133 s). SpaceEx mapping 0 approach guard 0.05 0 0.05 0.1 Θ s Θ 1 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 21 / 39

Applications Ensuring Safety for Complete Vehicle Control ➀ occupancy prediction ➁ trajectory planning controller ➃ trajectory tracking ➂ collision checking Matthias Althoff Analysis of Hybrid Systems 12. June 2015 22 / 39

Applications Consideration of Uncertainty reference trajectory obstacle Matthias Althoff Analysis of Hybrid Systems 12. June 2015 23 / 39

Applications Consideration of Uncertainty vehicle occupation reference trajectory obstacle Matthias Althoff Analysis of Hybrid Systems 12. June 2015 23 / 39

Applications Consideration of Uncertainty reachable set of the center reference trajectory obstacle Robust Safety Problem Is the planned maneuver of the autonomous vehicle still safe under uncertain initial states, uncertain measurements, and disturbances? Objective: Guarantee safety when bounds on uncertainties are known. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 23 / 39

Applications Consideration of Uncertainty reachable set of the center vehicle occupation possible collision Robust Safety Problem Is the planned maneuver of the autonomous vehicle still safe under uncertain initial states, uncertain measurements, and disturbances? Objective: Guarantee safety when bounds on uncertainties are known. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 23 / 39

Applications Online Verification Of Automated Driving lane change maneuver B lane change maneuver A y-position [m] 5 0 Test site Test vehicle initial occupancy obstacle final occupancy other vehicle ego vehicle reference trajectory ego vehicle (braking part) 5 20 0 20 40 60 80 100 120 x-position [m] Matthias Althoff Analysis of Hybrid Systems 12. June 2015 24 / 39

Applications Test Drive Results sx, sy [m] Ψ [rad] β [rad] δ [rad] v [m/s] δ y v x sx sy β Ψ x- and y-position orientation slip angle at center of mass front wheel angle velocity 0.5 0.5 0 0.5 lc B 2.5 lc A Ψ [rad] 3 Ψ [rad/s] Ψ [rad] Ψ [rad/s] 3 2.8 2.6 2.4 0.2 0 δ [rad] 0.2 lc B 0 lc A 0.5 0.2 0 δ [rad] 0.2 computation time: 1.8 times faster than maneuver time (Intel i7, 1.6GHz) Matthias Althoff Analysis of Hybrid Systems 12. June 2015 25 / 39

Applications Use Cases for Power Systems Transient stability analysis (specific fault) x 2 x 1 final set pre-fault set post-fault set Transient stability analysis (region of attraction) x 2 x 1 post-fault set pre-fault set Stability prediction under uncertain power demand and production voltage/phase reachable set allowed voltage/phase limits time t Matthias Althoff Analysis of Hybrid Systems 12. June 2015 26 / 39

Applications Abstraction of the Dynamic Model Original dynamic model (semi-explicit, index-1 DAEs) ẋ = f (x(t), y(t), u(t)) 0 = g(x(t), y(t), u(t)), [x T (0), y T (0)] T R(0), u(t) U, x R n d, y R na : differential & algebraic states, u R m : inputs, R(0): set of initial states, U: set of uncertain inputs Abstraction by a linear differential inclusion For t τ k = [t k, t k+1 ] (k: time step): x Ã(k) x Ũ(k), x R n d new differential states, Ũ: new set of uncertain inputs The algebraic states are extracted from the differential states (see later). Matthias Althoff Analysis of Hybrid Systems 12. June 2015 27 / 39

Applications IEEE 14-Bus Benchmark System 13 12 14 G 1 G 4 7 11 10 9 8 6 5 G G 2 3 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 28 / 39 G

Overall, the system has 14 dynamic state variables and 28 algebraic state variables. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 29 / 39 Applications Dynamic and Algebraic Equations The algebraic equations are obtained from the standard equations of a bus-network. Generator/Synchronous Condenser Dynamics The dynamics for each generator and synchronous condenser are described by the following set of equations: δ i = ω i ω 1 ω i = D i (ω i ω 1 ) + 1 T m,i 1 P g,i M i M i M i 1 Ṫ m,i = (ω i ω s ) 1 T m,i + 1 P c,i, T SV,i R D,i ω s T SV,i T SV,i M i [MJ/Hz 2 ] is the rotational inertia, D i [s/rad] the damping coefficient, T SV,i [s] is the time constant of the governor, and 1 R D,i [-] is the proportional gain of the governor.

Applications Reachable Set of the Dynamic Variables Black lines: random simulations; gray area: reachable set; white box: initial set ω2 378 377.5 377 376.5 376 375.5 375 374.5 0.6 0.4 0.2 δ 2 377 376.8 ω3 377.5 377 376.5 376 375.5 0.8 0.6 0.4 δ 3 2.045 2.04 ω4 377 376.5 376 0.8 0.6 0.4 δ 4 0.44 ω5 376.6 376.4 376.2 376 Tm,1 2.035 2.03 2.025 2.02 375.8 0.8 0.6 0.4 374 376 378 380 375 376 377 378 δ 5 ω 1 ω 2 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 30 / 39 Tm,2 0.435 0.43 0.425

Applications Reachable Set of the Algebraic Variables Black lines: random simulations; dark gray area: pre- and post-fault reachable set; light gray area: fault-on reachable set 0.3 0.3 0.4 0.4 0.4 0.5 Θ1 0.5 Θ2 0.5 Θ3 0.6 0.6 0.6 0.7 0.7 1.05 1.1 1.15 E 1 0.7 1.1 1.12 1.14 1.16 1.18 E 2 0.8 1.02 1.04 1.06 1.08 E 3 0.4 0.4 0.5 0.5 0.4 Θ4 0.6 Θ5 0.6 Θ7 0.5 0.7 0.7 0.6 0.8 0.8 0.7 1.08 1.09 1.1 1.12 1.122 1.124 1.015 1.02 1.025 E 4 E 5 V 7 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 31 / 39

Applications Verification of a Phase-Locked Loop (PLL) Digital phase-locked loop with charge pumps: CP i i v i Ref Φ ref Φ v phase frequency detector (PFD) UP DN C i ip v p1 R p3 v p VCO Φ out frequency divider 1/N C p1 R p2 C p3 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 32 / 39

Applications Hybrid Automaton Description of the PLL guard: Φ ref == 2π reset: Φ v := Φ v 2π Φ ref := 0 up active UP = 1, DW = 0 guard: Φ v == 0 reset: t := 0 both off UP = 0, DW = 0 guard: t == t d both active UP = 1, DW = 1 guard: Φ v == 2π reset: Φ ref := Φ ref 2π Φ v := 0 dw active UP = 0, DW = 1 guard: Φ ref == 0 reset: t := 0 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 33 / 39

Applications Hybrid Automaton Description of the PLL guard: Φ ref == 2π reset: Φ v := Φ v 2π Φ ref := 0 both off UP = 0, DW = 0 up active UP = 1, DW = 0 guard: t == t d dw active guard: Φ v == 0 reset: t := 0 both active UP = 1, DW = 1 Φ ref Φ v i i 2π 0 2π 0 I UP i t t 0 t on t d Matthias Althoff Analysis of Hybrid Systems 12. June 2015 33 / 39 t

Applications Continuous Dynamics of the PLL ẋ = Ax + Bu + c, 0 0 0 0 0 1 ( ) 0 1 1 C p1 R p2 + 1 C 1 R p3 C p1r p3 0 0 i 0 0 1 A = 1 0 C p3r p3 1 0 C p1 C p3r p3 0 0, B = 0 0 K i K N 0 p N 0 0 0 0, c = 0 0 2π N f 0 0 0 0 0 0 0 0 2πf ref Input values vary depending on the signals leaving the phase-frequency detector: [Ii UP, Ip UP ], if UP = 1, DW = 0 [Ii DW, Ip DW ], if UP = 0, DW = 1 u = [Ii UP + Ii DW, Ip UP + Ip DW ], if UP = 1, DW = 1 [0, 0], if UP = 0, DW = 0 Matthias Althoff Analysis of Hybrid Systems 12. June 2015 34 / 39

Specification Applications Transient Behavior Given any initial state and any valid set of parameters, verify that the locked condition ( Φ ref Φ v < Φ lock ) is reached in less than k cycles. Invariant Behavior Given a set of states in the locked condition, show that the locked condition is an invariant. phase difference Φ 0 0.1 0.2 0.3 transient part allowed Φ invariant part 0 500 1000 1500 cycle number Matthias Althoff Analysis of Hybrid Systems 12. June 2015 35 / 39

Applications Reachable Sets of the Phase-Locked Loop (first 200 cycles) 0.2 8 0.1 (Φv Φref)/2Π in [rad] 10 vp1 in [V] 6 4 2 0 2 4 0 0.1 0.2 0.3 0.4 0.35 0.4 0.45 0.5 0.55 v in [V] 0.6 0.5 0.65 0.35 0.4 i 0.1 (Φv Φref)/2Π in [rad] 0.2 8 6 vp in [V] 0.6 0.65 i 10 4 2 0 2 4 4 0.45 0.5 0.55 v in [V] 0 0.1 0.2 0.3 0.4 2 0 Matthias Althoff 2 4 vp1 in [V] 6 8 10 0.5 4 2 Analysis of Hybrid Systems 0 2 4 vp in [V] 6 8 10 12. June 2015 36 / 39

Computation Times Applications reachability analysis: avg. MATLAB simulation: Φ(0) total [s] 1 cycle [s] total [s] 1 cycle [s] [ 1, 0.8]π 55.0461 0.0270 48.3297 0.0237 [ 0.8, 0.6]π 54.4418 0.0275 47.9096 0.0242 [ 0.6, 0.4]π 53.4820 0.0280 46.2673 0.0242 [ 0.4, 0.2]π 47.8208 0.0264 44.4596 0.0245 [ 0.2, 0]π 42.9191 0.0260 38.5102 0.0233 Show videos... Matthias Althoff Analysis of Hybrid Systems 12. June 2015 37 / 39

Applications Further Reading Stability: M. Johansson: Piecewise Linear Control Systems A Computational Approach, Springer Lecture Notes in Control and Information Sciences no. 284, 2002. R.A. DeCarlo, M.S. Branicky, S. Pettersson, and B. Lennartsson: Perspectives and Results on the Stability and Stabilizability of Hybrid Systems, Proceedings of the IEEE, Vol 88, No. 7, 2000. Reachability: M. Althoff: Reachability Analysis and its Application to the Safety Assessment of Autonomous Cars, Technische Universität München, 2010. E. Asarin, T. Dang, G. Frehse, A. Girard, C. Le Guernic, O. Maler: Recent Progress in Continuous and Hybrid Reachability Analysis, Proc. of the IEEE Conference on Computer Aided Control Systems Design, 2006, pages 1582-1587. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 38 / 39

Conclusions Conclusions Stability: Switching between stable subsystems can destabilize a system. A common Lyapunov function is necessary and sufficient for arbitrarily switched systems. When the system is not arbitrarily switched, one often requires multiple Lyapunov functions to prove stability. Reachability: For most hybrid systems it is theoretically impossible to exactly compute the reachable set. Overapproximations of reachable sets can prove the correctness of hybrid systems. This is not possible with simulation techniques. Matthias Althoff Analysis of Hybrid Systems 12. June 2015 39 / 39