PhD Thesis Defense June 23, 2008

Transcription

1 1 Georgios E. Fainekos PhD Thesis Defense June 23, 2008 Department of Computer Information Science University of Pennsylvania fainekos at seas upenn edu

2 Embedded in : Automotive Systems 2 Latest BMW : 72 networked microprocessors 90% of innovation on electronics

3 Embedded in : Automotive Systems 3

4 Embedded in : Avionics 4 Boeing 777 : 1280 networked microprocessors 50% of design cost ($ time)

5 Embedded in : Medical Devices 5 Vision : Doctor-on-a-chip Operating room of the future Remote monitoring of elderly Digital hospital

6 Embedded in : Robotics 6 UAV UGV Bayraktar, Bayraktar, Fainekos, Fainekos, Experimental Experimental Cooperative Cooperative Control Control of of Fixed-Wing Fixed-Wing Unmanned Unmanned Aerial Aerial Vehicles, Vehicles, CDC CDC 07 07

7 Cyber-Physical Systems : Challenges 7 Composition Modularity Important for heterogeneous systems Robustness, Safety, Security Security cannot always be guaranteed, is system robust? Control Hybrid Systems Closing the loops at all levels Computational Abstractions New models programming languages Reconfigurable / Networked Systems Sensor networks embedded in critical infrastructure Model-based Development From models to platform-depended implementations Verification, Validation, Certification Safety-critical computing systems Particularly important for government agencies Education Training No education in cyber-physical systems Education in system architecture/integration?

8 Contributions of the thesis 8 A new notion of robustness for temporal logics Fainekos Robustness of temporal logic specifications, FATES/RV 2006 Fainekos Robustness of TL specifications for continuous time signals, TCS (Submitted) From discrete time to continuous time Fainekos Robust Sampling for MITL specifications, FORMATS 2007 Fainekos Robustness of TL specifications for continuous time signals, TCS (Submitted) Bounded time TL verification of dynamical systems Fainekos, Girard Temporal logic verification using simulation, FORMATS 2006 Fainekos MTL Robust Testing for LPV Systems, RTSS 2008 (Submitted) Hybrid automata synthesis from TL specifications Fainekos et al, Hierarchical Synthesis of Hybrid Controllers from TL Specifications, HSCC 2007 Fainekos et al, Temporal Logic Motion Planning for Dynamic Robots, Automatica (Accepted) Fainekos, Kress-Gazit Temporal Logic Motion Planning for Mobile Robots, ICRA 2005 Fainekos, Kress-Gazit Hybrid Controllers for Path Planning, CDC 2005

9 Talk Overview 9 Introduction Application areas Challenges Thesis Contributions Testing / Verification (Towards Certification) Problem Specification language (MTL) Robustness of Temporal Logic Specifications for signals From Discrete Time to Continuous Time From Signals to Systems Analog system robust testing / verification Hybrid system robust testing Final remarks Future work

10 Certification / Verification 10 Does the system satisfy the specification? Model-based code generation Cyber-Physical System System Specification Implementation Algorithm/Process YES NO Where how the system fails?

11 Example : Verifying a transmission line 11 V R1 L1 C1 R2 L2 D1 C2 Cc1 V1 D2 Cc2 V2 Rn Ln Cn Dn Ccn Vn System: x& ( t) = A x( t) + bu U out ( t) = Cx( t) Step input (t > 0): i i in ( t) Steady state at t=0: Property: Φ =Gp 1 F [0,0.85] G p 2 O (p 1 ) = [-1.5,1.5] O (p 2 ) = [0.8,1.2] Initial conditions: Τ Uncertain parameters e.g. Cœ[a 1,a 2 ]

12 What can we verify? 12 Verifying Cyber systems is a decidable problem The Turing award this year was given to the founders of model checking : E. Clark, A. Emerson J. Sifakis Cyber-System Specification Algorithm YES NO Applications in verification of software, hardware, protocols etc. Many software toolboxes : SPIN, SMV etc

13 What about hybrid (embedded) systems? 13 In general, verifying a hybrid system is undecidable R. Alur C. Courcoubetis N. Halbwachs T. A. Henzinger P.-H. Ho X. Nicollin A. Olivero J. Sifakis S. Yovine, The algorithmic analysis of hybrid systems, TCS Henzinger, Kopke, Puri, Varaiya, What's decidable about hybrid automata? Proceedings of the twenty-seventh annual ACM symposium on Theory of computing. Cyber-Physical System System Specification Algorithm YES NO

14 What can be done? 14 Previous approaches to the undecidability problem : identifying decidable classes : Alur, Henzinger, Lafferriere, semi-decidable algorithms : Krogh, Alur, Henzinger, Dang, Ivančić, Girard, Mitchell, Tomlin, Maler, barrier certificates : Prajna, Jadbabaie, systematic simulations / model based testing : Krogh, Maler, Dang, Lee, Sokolsky, Kumar, Esposito, LaValle, Vardi, In Practice : Simulations! Schuller, Brangs, Rothfuß, Lutz, Breit: Development methodology for dynamic stability control systems, International Journal of Vehicle Design Vol. 28, No.1/2/3 pp Gielen, Rutenbar: Computer-Aided Design of Analog Mixed-Signal Integrated Circuits, Proceedings of the IEEE, Vol. 88, No. 12, 2000

15 ROBUST SIMULATIONS 15 Reach(I) I Advantages : A finite number of simulations Coverage guarantees Scales as well as simulation scales More complicated specifications than safety Almost no parameters to set besides the simulation parameters New tools : we need metrics

16 Talk Overview 16 Introduction Application areas Challenges Thesis Contributions Testing / Verification (Towards Certification) Problem Specification language (MTL) Robustness of Temporal Logic Specifications for signals From Discrete Time to Continuous Time From Signals to Systems Analog system robust testing / verification Hybrid system robust testing Final remarks Future work

17 Metric Temporal Logic (MTL) 17 until release Syntax: Φ ::= T ^ p Ÿ p Φ 1 Φ 2 Φ 1 Φ 2 Φ 1 U I Φ 2 Φ 1 R I Φ 2 I can be of any bounded or unbounded interval of +, but I «i.e. I = [0,+ ), I = [2.5,9.8] Derived operators: Eventually (in the future) Always (globally) F I Φ := T U I Φ G I Φ := ^R I Φ Koymans 90, Specifying real-time properties with metric temporal logic

18 LTL intuition 18 G a -always a a a a a a a F a eventually a * * * a * * X a next state a * a * * * * a U b a until b a B b a before b a * a a b * * * a * b *

19 MTL : An example for signals 19 a t I time F I a I a Boolean abstraction t I time

20 Temporal Logic Testing 20 LTL / MTL Φ =Gp 1 F [0,Τ] G p 2 A/D Boolean abstraction Monitoring Algorithm Truth Value {^,T} [Maler Nickovic 04] [Thati Rosu 04] [Rosu Havelund 05] [Geilen 01] others

21 Talk Overview 21 Introduction Application areas Challenges Thesis Contributions Testing / Verification (Towards Certification) Problem Specification language (MTL) Robustness of Temporal Logic Specifications for signals From Discrete Time to Continuous Time From Signals to Systems Analog system robust testing / verification Hybrid system robust testing Final remarks Future work

22 Two signals that satisfy the same spec, but 22 p s 1 MTL MTL Spec: Spec: G(p G(p 1 YF 1 YF 2 p 2 2 ) 2 ) s p 1

23 LTL to motion planning 23 F(p F(p 2 2 F(p F(p 3 3 F(p F(p 4 4 (p (p 2 2 p 3 ) 3 ) U p 1 ))) 1 ))) p 1 p 2 p 1 p 2 p 3 p 4 p 3 p 4 Fainekos, Fainekos, Kress-Gazit Kress-Gazit Temporal Temporal Logic Logic Motion Motion Planning Planning for for Mobile Mobile Robots, Robots, ICRA ICRA Fainekos, Fainekos, Kress-Gazit Kress-Gazit Hybrid Hybrid Controllers Controllers for for Path Path Planning, Planning, CDC CDC

24 Robustness of Temporal Logics 24 LTL / MTL Φ =G(p 1 YF 2 p 2 ) Monitor/Tester Robustness parameter ε»{± } ε ε Fainekos Fainekos Robustness Robustness of of temporal temporal logic logic specifications, specifications, FATES/RV FATES/RV Fainekos Fainekos Robustness Robustness of of temporal temporal logic logic specifications specifications for for signals, signals, Submitted Submitted to to TCS, TCS,

25 Related Research 25 Robustness in temporal logics WRT time : Huang, Voeten, Geilen 03, Real-time property preservation in approximations of timed systems Henzinger, Majumdar, Prabhu 05, Quantifying similarities between timed systems WRT state : de Alfaro, Faella, Stoelinga 04, Linear Branching Metrics for Quantitative Transition Systems Lamine, Kabanza 00, Using fuzzy TL for monitoring behavior-based mobile robots de Alfaro, Faella, Henzinger, Majumdar, Stoelinga 04, Model Checking Discounted Temporal Properties Huth, Kwiatkowska 97, Quantitative analysis model checking

26 (Signed) Distance 26 Let x X be a point, C X be a set d be a metric. Then we define dist d (x, C) := inf{d(x, y) y cl(c)} depth d (x, C):=dist d (x, X\C) Dist d (x, C):= è à dist d (x, C) depth d (x, C) if x 6 C if x C X dist d (x,c) x x x C depth d (x,c) B d (x,ε)

27 Definition of robustness for signals 27 Given a signal s, we can define the robustness degree as ε ε := := Dist Dist ρ (s, ρ (s, L(Φ)) ρ(s,s ) = sup {d(s(t),s (t)) t œ R} s L(ŸΦ) X R s L(Φ)

28 Main result 28 Theorem: Let LetΦ be be an an MTL MTLformula, ssbe be a (continuous or or discrete time) time) signal signal ε >0 ε >0be be the therobustness parameter of of Φ with with respect to to s, s, then then for for all all s s in in B ρ (s,ε) ρ (s,ε) we we have have that that s s Φ iff iff s s Φ Fainekos Fainekos Robustness Robustness of of temporal temporal logic logic specifications, specifications, FATES/RV FATES/RV Fainekos Fainekos Robustness Robustness of of temporal temporal logic logic specifications specifications for for signals, signals, Submitted Submitted to to Theoretical Theoretical Computer Computer Science, Science, Fainekos Fainekos Robustness Robustness of of temporal temporal logic logic specifications specifications for for finite finite timed timed state state sequences sequences in in metric metric spaces, spaces, Technical Technical Report Report MS-CIS-04-32, MS-CIS-04-32,

29 Software toolbox : TaLiRo Available at : 29 Input: Discrete time signal Input: LTL / MTL formula & Observation map Monitor/Tester Output: ε œ»{± }

30 Intuition Example 30 Specification : Fp = T U p O(p) X

31 Running TaLiRo on a Dell PowerEdge O(p 1 ) = [1.5,+ ) s(i) = sin τ(i)+sin 2τ(i) τ(i) = 0.2i Time G(p 1 Ø F (0.0,1.0) Ÿp 1 )

32 Guaranteed correctness under noise uncertainty 34 F I2 (p I2 (p 2 2 F I3 (p I3 (p F 3 3 I4 (p I4 (p 4 4 (p (p 2 2 p 3 ) 3 ) U I1 p I1 1 ))) 1 ))) 6 Assume: sensor sensor accuracy ±0.1 ±0.1 6 Robustness estimate ε ε =

33 Talk Overview 35 Introduction Application areas Challenges Thesis Contributions Testing / Verification (Towards Certification) Problem Specification language (MTL) Robustness of Temporal Logic Specifications for signals From Discrete Time to Continuous Time From Signals to Systems Analog system robust testing / verification Hybrid system robust testing Final remarks Future work

34 From Discrete to Continuous Time 36 MTL formula Signal dynamics Sampling Conditions TL TL Robustness Bounds on on the the continuous time time robustness estimate YES NO Fainekos Fainekos Robust Robust Sampling Sampling for for MITL MITL specifications, specifications, FORMATS FORMATS Fainekos Fainekos Robustness Robustness of of TL TL specifications specifications for for continuous continuous time time signals, signals, TCS TCS (Submitted) (Submitted)

35 From Signals to Systems 37 Given a system Σ, we can define the robustness degree as ε ε := := dist dist ρ (L(Σ), ρ L(ŸΦ)) = inf inf {ρ(s,s ) ssœ L(Σ), s s œ L(ŸΦ)} ρ(s,s ) = sup {d(s(t),s (t)) t œ R} L(Φ) ε L(Σ) L(ŸΦ) X R Fainekos Fainekos MTL MTL Robust Robust Testing Testing for for LPV LPV Systems, Systems, RTSS RTSS (Submitted) (Submitted)

36 Talk Overview 38 Introduction Application areas Challenges Thesis Contributions Testing / Verification (Towards Certification) Problem Specification language (MTL) Robustness of Temporal Logic Specifications for signals From Discrete Time to Continuous Time From Signals to Systems Analog system robust testing / verification Hybrid system robust testing Final remarks Future work

37 Example : a study of transient dynamics 39 System (dim 81): Step input (t > 0): Steady state at t=0 - : Property: Φ =Gp 1 F [0,Τ] G p 2 O (p 1 ) = [-1.5,1.5] O (p 2 ) = [0.8,1.2] Initial conditions: Τ

38 Robust TL testing of analog systems 40 Closed-loop system Σ xç =f(x) X 0 X y = g(x) LTL / MTL Φ =G p 1 F [0,Τ] G p 2 Simulator // Tester Tester B ρ (L(Σ),δ) L(Φ) Fainekos, Fainekos, Girard Girard Temporal Temporal logic logic verification verification using using simulation, simulation, FORMATS FORMATS Fainekos Fainekos MTL MTL Robust Robust Testing Testing for for LPV LPV Systems, Systems, RTSS RTSS (Submitted) (Submitted)

39 Related Research 41 Verification/Testing of Analog Systems (using TL) Hartong, Hedrich, Barke 02, On Discrete Modeling Model Checking for Nonlinear Analog Systems Ghosh, Vemuri 99, Formal Verification of Synthesized Analog Designs Frehse, Krogh, Rutenbar, Maler 05, Time Domain Verification of Oscillator Circuit Properties Gupta, Krogh, Rutenbar 04, Towards Formal Verification of Analog Designs Donze, Maler 07, Systematic Simulation using Sensitivity Analysis

40 Main idea 42 Closed-loop system Σ: xç =f(x) X 0 X y = g(x) Specification Φ L(Σ) L(Φ) time Proj 0 (P Φ L(Σ)) X 0 ε robustness parameter B ρ (σ, ε )

41 Achieving coverage I 43 Closed-loop system Σ: xç =f(x) X 0 X y = g(x) Specification Φ L(Σ) L(Φ) time Proj 0 (P Φ L(Σ)) X 0 Good news! Coverage with a finite number of simulations

42 Computing bisimulation functions 44 Quadratic Bisimulation Functions for Deterministic Linear Systems xç =Ax y = Cx y V(x) = is a bisimulation function if A T x M C T T Mx C M + MA 0 Bisimulation Functions using Sum Of Squares Relaxation xç =f(x) y = g(x) y V(x1, x 2) = q(x1, x 2) is a bisimulation function if q(x1, x 2) g1(x1) g 2 (x 2 ) 2 is SOS q(x1, x x 1 2 ) q(x1, x f1(x1) x 2 2 ) f 2 (x 2 ) is SOS

43 Achieving coverage II 45 Closed-loop system Σ: xç =f(x) X 0 X y = g(x) Specification Φ L(Σ) L(Φ) time Proj 0 (P Φ L(Σ)) X 0 Even better news! It is possible to verify the system with just one simulation

44 Quick falsification 46 Closed-loop system Σ: xç =f(x) X 0 X y = g(x) Specification Φ L(Σ) L(Φ) time Proj 0 (P Φ L(Σ)) Observation! A robust system with respect to the property requires less simulations X 0

45 Coverage certificates 47 Closed-loop system Σ: xç =f(x) X 0 X y = g(x) Specification Φ L(Σ) L(Φ) time Proj 0 (P Φ L(Σ)) X 0 Good news! We get coverage guarantees after K iterations.

46 Main results 48 Theorem: Let Let V be be a bisimulation function, let let (x (x 1,y 1,y 1 ) 1 ) be be a trajectory of of Σ, Σ, εεbe be the the robustness parameter of of Φ wrt wrty 1 1 δ>0, δ>0, then for for any any other trajectory (x (x 2,y 2,y 2 ) 2 ) such such that that V(x V(x 1 (0),x 1 (0),x 2 (0)) 2 (0)) < ε-δ ε-δimplies that that the the robustness parameter of of Φ wrt wrty 1 is 1 is greater or or equal to to δ. δ. Proposition: Let Let V be be a bisimulation function. For For any any compact set set of of initial conditions X 0 0,, for for all all ζ ζ >0, >0, there exists a finite set set of of points {x {x 1,,x 1,,x r } r X 0 such 0 such that that for for all all x X 0, 0, there exists x i, i, such such that that V(x,x i ) i ) ζζ Fainekos, Fainekos, Girard Girard Temporal Temporal logic logic verification verification using using simulation, simulation, FORMATS FORMATS Fainekos, Fainekos, Robustness Robustness of of Temporal Temporal Logic Logic Specifications, Specifications, PhD PhD Thesis Thesis

47 Main results 49 Theorem: Let Let (x (x 1,y 1,y 1 ),,(x 1 r,y r,y r ) r ) be be trajectories of of Σ such such that thatdisc(x 0,ζ)= 0,ζ)= {x {x 1 (0),,x 1 r (0)}. r (0)}. Let Letεε i be i be the the robustness parameter of of Φ wrt wrty i.then, i "i œ {1,,r}.ε i > ζ+δ ï B ρ (L(Σ),δ) L(Φ) Proposition: If If Σ 1 is 1 is δ-approximately simulated by by Σ 2 2 B ρ (L(Σ 2 ),δ) L(Φ) then L(Σ 1 ) L(Φ) Fainekos, Fainekos, Girard Girard Temporal Temporal logic logic verification verification using using simulation, simulation, FORMATS FORMATS Fainekos, Fainekos, Robustness Robustness of of Temporal Temporal Logic Logic Specifications, Specifications, PhD PhD Thesis Thesis Fainekos Fainekos MTL MTL Robust Robust Testing Testing for for LPV LPV Systems, Systems, RTSS RTSS (Submitted) (Submitted)

48 Verification example 50 Consider the linear system with dynamics A = C = with initial conditions X 0 = [0.4,0.8] x [-0.3,-0.1] Verify that it satisfies the specification Φ =G p 1 G [8, ] p 2 where O(p 1 ) = x [-0.6,0.6] O(p 2 ) = [-0.4,0.4] x [-0.4,0.4] with robustness δ = over the time domain [0,14] Time

49 Verification example - Initialization 51 Compute the bisimulation function V(x) = x T Mx A T M M + C T C MA 0 M = Choose as first point the center x c of the hyper-rectangle Compute the maximum value of the bisimulation function of all the points in the set X 0 relative to x c

50 Verification example First test y (2) y (1)

51 Verification example 2 nd iteration y (2) y (1)

52 Verification example 3 rd iteration y (1) (t) y (2) (t) y (2) Time y (1)

53 55 θ Experimental Results (MATLAB toolbox) Property: T Φ =Gp 1 F [0,Τ] G p 2 O (p 1 ) = [-θ,θ], O (p 2 ) = [0.8,1.2] θ T=0.8 T=1.2 T=1.6 θ=1.4 False 1 False 1 False 1 θ=1.5 False 1 False 15 False 13 T θ=1.6 False 1 True 17 True 7 from Zhi Han s PhD Thesis 2005

54 56 θ Experimental Results (MATLAB toolbox) Property: T Φ =Gp 1 F [0,Τ] G p 2 O (p 1 ) = [-θ,θ], O (p 2 ) = [0.8,1.2] θ T=0.8 T=1.2 T=1.6 θ=1.4 False 1 False 1 False 1 θ=1.5 False 1 False 15 False 13 T θ=1.6 False 1 True 17 True from Zhi Han s PhD Thesis 2005

55 57 θ Experimental Results (MATLAB toolbox) Property: T Φ =Gp 1 F [0,Τ] G p 2 O (p 1 ) = [-θ,θ], O (p 2 ) = [0.8,1.2] θ T=0.8 T=1.2 T=1.6 θ=1.4 False 1 False 1 False θ=1.5 False 1 False 15 False 13 θ=1.6 False 1 True 17 True 7 from Zhi Han s PhD Thesis 2005

56 θ Experimental Results (MATLAB toolbox) 58 Property: T Φ =Gp 1 F [0,Τ] G p 2 O (p 1 ) = [-θ,θ], O (p 2 ) = [0.8,1.2] θ T=0.8 T=1.2 T=1.6 θ=1.4 False 1 False 1 False 1 θ=1.5 False 1 False 15 False 13 θ=1.6 False 1 True 17 True 7 from Zhi Han s PhD Thesis 2005

57 1.6 θ Experimental Results (MATLAB toolbox) Property: T Φ =Gp 1 F [0,Τ] G p 2 O (p 1 ) = [-θ,θ], O (p 2 ) = [0.8,1.2] Uout(Volt) θ T=0.8 T=1.2 T= θ=1.4 θ=1.5 False False 1 1 False False Time(nsec) 1 15 False True θ= False 1 False 13 True 7 from Zhi Han s PhD Thesis 2005

58 Talk Overview 60 Introduction Application areas Challenges Thesis Contributions Testing / Verification (Towards Certification) Problem Specification language (MTL) Robustness of Temporal Logic Specifications for signals From Discrete Time to Continuous Time From Signals to Systems Analog system robust testing / verification Hybrid system robust testing Final remarks Future work

59 How robust is a hybrid test trajectory? 61 L 1 Init Unsafe L 2 Julius, Julius, Fainekos, Fainekos, An, An, Lee, Lee, Robust Robust Test Test Generation Generation Coverage Coverage for for Hybrid Hybrid Systems, Systems, HSCC HSCC

60 Talk Overview 62 Introduction Application areas Challenges Thesis Contributions Testing / Verification (Towards Certification) Problem Specification language (MTL) Robustness of Temporal Logic Specifications for signals From Discrete Time to Continuous Time From Signals to Systems Analog system robust testing / verification Hybrid system robust testing Final remarks Future work

61 Main Contributions 63 Testing / Verification (toward Certification) Robust temporal logic testing Semantics which maintains topological information Applications in signal testing/monitoring Analog system robust temporal logic testing Combining logic with system dynamics Applications in analog system verification Hybrid system robust testing Robustness wrt to switching conditions Applications in embedded systems mixed-signal circuits From discrete time to continuous time How specification can guide sampling? Automated Synthesis of hybrid controllers Non-reactive planning (bottom up) Kinematic & dynamic model motion planning Reactive planning (bottom up) Distributed multi-robot planning Top-down hybrid system synthesis Human-Robot interfaces UAV testbed platform development

62 65 Thank you!! Acknowledgements: Rajeev Alur, Madhukar An, Selcuk Bayraktar, Edmund M. Clarke, Antoine Girard, A. Agung Julius, Hadas Kress-Gazit, Insup Lee, Savvas G. Loizou, George J. Oleg Sokolsky Funding: UPenn, NSF EHS , NSF ITR , ARO MURI DAAD