Models for Control and Verification

Transcription

1 Outline Models for Control and Verification Ian Mitchell Department of Computer Science The University of British Columbia Classes of models Well-posed models Difference Equations Nonlinear Ordinary Differential Equations Syntax vs semantics Visualization Optimal Control Objective and value functions Verification: Reachability Forward & backward, tubes & sets, maximal & minimal research supported by National Science and Engineering Research Council of Canada January 2009 Ian Mitchell (UBC Computer Science) 2 Control & Verification Require Modeling Dynamic systems change with time We wish to reason about that change Control: We seek to guide the evolution to achieve a desired objective Verification: We seek to confirm the evolution will achieve a desired objective Control and verification require prediction of future evolution Prediction is achieved by mathematical models System is described by state and time Discrete variable Discrete vs Continuous Drawn from a countable domain, typically finite Often no useful metric other than the discrete metric Often no consistent ordering Examples: names of students in this room, rooms in this building, natural numbers, grid of R d, Continuous variable Drawn from an uncountable domain, but may be bounded Usually has a continuous metric Often no consistent ordering Examples: Real numbers [ 0, 1 ], R d, SO(3), January 2009 Ian Mitchell (UBC Computer Science) 3 January 2009 Ian Mitchell (UBC Computer Science) 4 1

2 Classes of Models for Dynamic Systems Discrete time and state Continuous time / discrete state Discrete event systems Discrete time / continuous state Continuous time and state Markovian assumption All information relevant to future evolution is captured in the state variable Deterministic assumption Future evolution completely determined by initial conditions Not the only classes of models Well-Posed Models Mathematical models may not behave nicely May describe impossible evolutions May not be easy to apply formal reasoning We want to forbid such (eg ignore) models Common desirable traits There exists a solution for all (or some) time The solution is unique The solution depends continuously on the data (initial conditions, dynamics) January 2009 Ian Mitchell (UBC Computer Science) 5 January 2009 Ian Mitchell (UBC Computer Science) 6 Difference Equations Lipschitz Continuity Existence: for all t and x, f(t, x) Uniqueness: for all t and x, f(t, x) = 1 Continuous dependence on the data: for all t, x, y there exists constant λ such that Only makes sense if state space has a continuous metric Sufficient but not necessary Might also want to handle mistakes in f Called Lipschitz continuity with respect to x (or y) Constant λ is the Lipschitz constant Relationship with continuity and differentiability? Continuity Differentiability Differentiability with bounded derivative no relation Lipschitz continuity Lipschitz continuity Lipschitz continuity January 2009 Ian Mitchell (UBC Computer Science) 7 January 2009 Ian Mitchell (UBC Computer Science) 8 2

3 Lipschitz Continuous Functions Ordinary Differential Equations (ODEs) Which of these functions is Lipschitz continuous? What about second order ODE? Newton s second law: force = (mass)(acceleration) A B Need to reformulate into first order form Define new variable z(t) R 2*d C D May also be useful to remove dependence on t Define new variable y(t) R d+1 Called autonomous system in mathematics January 2009 Ian Mitchell (UBC Computer Science) 9 January 2009 Ian Mitchell (UBC Computer Science) 10 Well-Posed ODEs Consider initial value problem (IVP): x(t i ) = x i If f is Lipschitz continuous in x for all t [ t i, t f ] Ill-Posed ODEs Why do we care that the ODE is well-posed? Theory: much depends on the existence of a unique solution Numerics: approximate solution may not be desired solution, and may not even be near a true solution There exists a unique solution x(t) for t [ t i, t f ] for each x i such that dx/dt dt exists and dx/dt dt = f(t,x) For perturbed initial data y i yielding y(t) For perturbed dynamics Sufficient but not necessary conditions January 2009 Ian Mitchell (UBC Computer Science) 11 January 2009 Ian Mitchell (UBC Computer Science) 12 3

4 Hybrid Systems Some systems are not purely continuous or discrete Computer control of continuous systems Vastly varying timescales for continuous systems, in which only the equilibrium value of the faster timescale components matters Simplify some components of a system by discretizing a continuous state or taking continuum limit of discrete state Hybrid Automata Discrete modes and transitions Continuous evolution within each mode No simple conditions for well-posed HA seven mode collision avoidance protocol January 2009 Ian Mitchell (UBC Computer Science) 13 January 2009 Ian Mitchell (UBC Computer Science) 14 Checking a Model Well-posed conditions are examples of static checks: tests applied directly to the model Model does not itself evolve, but is a static entity Complexity of check depends only on the complexity of the model Alternative: Dynamic checks Requires understanding the evolving solution Complexity of check depends on the complexity of the solution trajectory Restricted Classes of Model Many results in control and verification assume a restricted class of models Permits more checks to be static May simplify checks of dynamic Example: FA and ODEs both have easy static checks for well-posedness; hybrid systems do not Our nonlinear ODE and DI models are very general Most of what we discuss (beyond well-posedness) will be semantic / dynamic checks January 2009 Ian Mitchell (UBC Computer Science) 15 January 2009 Ian Mitchell (UBC Computer Science) 16 4

5 phase Space Most of the visualization of system evolution will be done in the phase or state space (ignore time) Pendulum states angle and angular velocity Visualization January 2009 Ian Mitchell (UBC Computer Science) 17 state vs time pendulum workspace Classes of models Outline Well-posed models Difference Equations Nonlinear Ordinary Differential Equations Syntax vs semantics Visualization Optimal Control Objective and value functions Verification: Reachability Forward & backward, tubes & sets, maximal & minimal Dimitri Bertsekas, Dynamic Programming & Optimal Control, Athena Scientific (3 rd edition 2005) January 2009 Ian Mitchell (UBC Computer Science) 18 Achieving Desired Behaviours We can attempt to control a system when there is a parameter u of the dynamics (the control input ) which we can influence Time dependent dynamics are possible, but we will mostly deal with time invariant systems Visualization: Vector Fields Introduction of a free control input changes the vector field plot in the phase space into a field of cones (nondeterministic) Feedback control law changes it back into a (static) vector field Open loop control law does not unspecified input signal Without a control signal specification, system is nondeterministic Current state cannot predict unique future evolution Control signal may be specified Open-loop u(t) or u: R U Feedback, closed-loop u(x(t)) or u: S U Either choice makes the system deterministic again no inputs ( autonomous for control engineers) feedback input signal January 2009 Ian Mitchell (UBC Computer Science) 19 January 2009 Ian Mitchell (UBC Computer Science) 20 5

6 Objective Function We distinguish quality of control by an objective / payoff / cost function, which comes in many different variations eg: discrete time discounted with fixed finite horizon t f Value Function Choose input signal to optimize the objective Optimize: cost is usually minimized, payoff is usually maximized and objective may be either Value function is the optimal value of the objective function May not be achieved for any signal (eg: min should be inf) eg: continuous time no discount with target set T Set of signals U is contentious For implementation purposes, we desire restricted classes: bounded, continuous, piecewise constant Unfortunately, theory applies to (and thus can only guarantee optimality with) very general classes: measurable January 2009 Ian Mitchell (UBC Computer Science) 21 January 2009 Ian Mitchell (UBC Computer Science) 22 Example: LQR for Linear Systems Much of the optimal control literature and most classes focus (without mentioning it) on linear systems Corresponding objective functions are usually quadratic where A, B, Q, R, Q f are all matrices of appropriate size Successful but restricted class of problems Not rigorously part of the results to follow (due to a technicality) January 2009 Ian Mitchell (UBC Computer Science) 23 Classes of models Outline Well-posed models Difference Equations Nonlinear Ordinary Differential Equations Syntax vs semantics Visualization Optimal Control Objective and value functions Verification: Reachability Forward & backward, tubes & sets, maximal & minimal Ian Mitchell, Comparing Forward and Backward Reachability as Tools for Safety Analysis, Hybrid Systems Computation and Control, LNCS 4416, Springer-Verlag (2007). January 2009 Ian Mitchell (UBC Computer Science) 24 6

7 Verification: Safety Analysis Does there exist a trajectory of system H leading from a state in initial set I to a state in terminal set T? (under some policy for input u( )) Typical Systems: ODEs Common model for continuous state spaces Standard existence and uniqueness January 2009 Ian Mitchell (UBC Computer Science) 25 January 2009 Ian Mitchell (UBC Computer Science) 26 Typical Systems: Hybrid Automata Adapted from [Gao, Lygeros & Quincampoix 2006] Challenging existence and uniqueness eg: [Broucke & Arapostathis, Sys. & Con. Letters 2002] or [Lygeros, Johansson, Simic, Zhang & Sastry, TAC 2003] requires at least non-zeno and non-blocking all non-determinism must be expressed through input u( ) Working with Sets Optimal control works with a single optimal trajectory Verification works with sets of trajectories Takes a nondeterministic (but not probabilistic) viewpoint Basic construct is reachability Many versions: forward and backward, sets or tubes What should the input do? Many related concepts in control theory Invariant sets, controlled invariant sets, stability Safety is not the only verification goal Liveness is a common goal, but often harder to verify January 2009 Ian Mitchell (UBC Computer Science) 27 January 2009 Ian Mitchell (UBC Computer Science) 28 7

8 Forward Reachability Start at initial conditions and compute forward Backward Reachability Start at terminal set and compute backwards January 2009 Ian Mitchell (UBC Computer Science) 29 January 2009 Ian Mitchell (UBC Computer Science) 30 Exchanging Algorithms Algorithms are (mathematically) interchangeable if system dynamics can be reversed in time Maximal Reachability Input signal u( ) maximizes size of the set or tube For example: Then January 2009 Ian Mitchell (UBC Computer Science) 31 January 2009 Ian Mitchell (UBC Computer Science) 32 8

9 Maximal Reachability Definition Maximal Reachability Results Reach sets and tubes provide similar information The following properties are equivalent Any maximal reachability operator can be used to demonstrate safety for all possible inputs January 2009 Ian Mitchell (UBC Computer Science) 33 January 2009 Ian Mitchell (UBC Computer Science) 34 Maximal Reachability Demonstration Maximal Reachability Demonstration Forward Reach Set Results Forward Reach Tube Results January 2009 Ian Mitchell (UBC Computer Science) 35 January 2009 Ian Mitchell (UBC Computer Science) 36 9

10 Maximal Reachability Demonstration Maximal Reachability Demonstration Backward Reach Set Results Backward Reach Tube Results January 2009 Ian Mitchell (UBC Computer Science) 37 January 2009 Ian Mitchell (UBC Computer Science) 38 Minimal Reachability Input signal u( ) minimizes size of the set or tube Minimal Reachability Definition January 2009 Ian Mitchell (UBC Computer Science) 39 January 2009 Ian Mitchell (UBC Computer Science) 40 10

11 Minimal Reachability Results Reach tubes provide more information Minimal Reachability Results Backward reach tubes are the only minimal reachability operator that can prove that there exists an input u( ) which keeps the system safe Choice of trajectory length t is quantified first for sets but last for tubes Basic problem with minimal forward reachability: the state lying in the terminal set is chosen before the input, while the state lying in the initial set is chosen after January 2009 Ian Mitchell (UBC Computer Science) 41 January 2009 Ian Mitchell (UBC Computer Science) 42 Minimal Reachability Demonstration Minimal Reachability Demonstration (Correct) Backward Reach Tube Results (Incorrect) Forward Reach Tube Results January 2009 Ian Mitchell (UBC Computer Science) 43 January 2009 Ian Mitchell (UBC Computer Science) 44 11