DryVR: Data-driven verification and compositional reasoning for automotive systems

Transcription

1 DryVR: Data-driven verification and compositional reasoning for automotive systems Chuchu Fan, Bolun Qi, Sayan Mitra, Mahesh Viswannathan University of Illinois at Urbana-Champaign CAV 2017, Heidelberg, Germany

2 Hybrid modelling: theory vs. reality Control systems in textbook Control system in reality dx dt = f x, u ; u = g(x) 2

3 All models are wrong, some are useful Dry R Gain serenity to accept models as they are 3

4 A new view of knowledge in hybrid models Complete information of switching structure Executable access to mode dynamics DryVR s Executable hybrid model + = 4

5 DryVR model of lane merge Cruise Cruise Accelerate Accelerate Decelerate Accelerate Turn_Right Accelerate 1 7 Cruise Cruise 5 Accelerate Cruise 6 Turn_Right Cruise 5

6 DryVR model semantics [a, b] + = Transition graph Trace: l., t., l /, t /,, l 1 Black-box simulator Trajectory: τ(t) Labeled trajectory set: τ, l TL Hybrid system H = L, Θ, G, TL State: a point in R : L Reach = x, l for some v, t, x, l is reachable from Θ} Reach v: all states reachable in vertex v 6

7 Outline Proof rules Bounded model checking Case studies 7

8 Composition for unbounded time analysis If Reach B Reach A then G. G / = G. G / G. G / M A B 8

9 Composition for unbounded time analysis If Reach B Reach A then G. G / G. G / M Reach A B Reach 9

10 Reasoning about behavior containment Trace containment G. G / Trajectory containment TL. TL / If Θ. Θ /, G. G /, TL. TL /, then G. G / Reach Reach 10

11 Simulation-driven bounded verification Safety problem: given initial set Θ and unsafe set U, decide Reach U =? τ t τ(0) Θ Reach Unsafe time 11

12 Simulation-driven bounded verification Simulation-driven verification for a single vertex v Simulate Generalization Check and refine Discrepancy β bounds distance between neighboring trajectories τ. (t) τ / (t) β(τ. (0), τ / (0), t), τ. τ \ τ / Θ From a single simulation of τ. (t) and discrepancy β overapproximate the reach set from a neighborhood of τ. 0 Unsafe Earlier approaches use f x, YZ [ [Duggirala et al. TACAS Y [ 15] [Fan et al. CAV 15-16] inapplicable time 12

13 Learning discrepancy from data Global exponential discrepancy function β x., x /, t = x. x / Ke^_ For any pair of trajectories τ. and τ / in mode t 0, T, τ. t τ / t τ. 0 τ / 0 Ke^_ Taking logarithm and rearrange: t, ln τ. t τ / (t) τ. 0 τ / (0) γt + ln K 13

14 Learning linear separators For a subset S R R, a linear separator is a pair a, b R / such that x, y S, x ay + b Algorithm: 1. Draw k pairs x., y.,, (x 1, y 1 ) from S according to D. 2. Find a, b R / such that x M ay M + b for all i {1,, k}. Proposition [Valiant 84]: Let ε, δ R p. If k ln then with probability ṡ ṫ 1 δ, the above algorithm finds (a, b) such that err D a, b < ε. err D a, b = D( x, y S x > ay + b}) 14

15 Learning discrepancy from data Solve the LP problem: min 2c ln K + c c + 1 γt --- Volume of the reach set s. t. i, j, s, ln _ ƒ _ (\) ƒ (\) γt + ln K 1 million testing show 96% accuracy for 10 training trajectories, and >99.9% for 20 Other discrepancy shapes in paper: piece-wise exponential, global polynomial, piece-wise polynomial 15

16 Bounded safety algorithm 1. Compute reach set from Θ: proceeds on G in a topologically sorted order 2. Refinement: Split Θ to smaller sets Split transition time interval to smaller intervals Guarantee: Assuming that the learned discrepancy function is correct: Soundness Relative completeness Θ Restrict to [c, d] Restrict to [a, b] [a, b] 1 7 [c, d] 5 6 Discrepancy has err D a, b < ε with ṡ ln ṫ samples 16

17 Automotive maneuvers Model Time horizo n Unsafe set # Refinement Safe Run time Auto-passing Lane-merge 50 Collision 4 208s 50 Collision 5 152s 50 Collision 0 55s 50 Collision 0 38s Lane-mergehighway Powertrain 80 Automatic transmission 50 Collision 4 197s 50 Collision 0 21s 50 Air/Fuel out of bound Engine speed too high 2 217s 2 109s Reach set of positions 17 time

18 Case studies: Engine control Model Time horizo n Unsafe set # Refinement Safe Run time Auto-passing Lane-merge 50 Collision 4 208s 50 Collision 5 152s 50 Collision 0 55s 50 Collision 0 38s Start up Lane-mergehighway Powertrain 80 Automatic transmission 50 Collision 4 197s 50 Collision 0 21s 50 Air/Fuel out of bound Engine speed too high 2 217s 2 109s Sensor fail Normal Power [Jin et al. HSCC 14] 18

19 Case studies: transmission control Model Time horizo n Unsafe set # Refinement Safe Run time Auto-passing 50 Collision 4 208s 50 Collision 5 152s Lane-merge Lane-mergehighway 50 Collision 0 55s 50 Collision 0 38s 50 Collision 4 197s 50 Collision 0 21s Gear 1 Gear 2 Gear 3 Gear 4 Gear 5 Powertrain 80 Air/Fuel out of bound 2 217s Automatic transmission 50 Engine speed too high 2 109s 19

20 Conclusion A fresh perspective (DryVR s model) on modeling hybrid systems white box transition graph + black box simulator Case studies ADAS / AV Enables types of static-dynamic analysis Black-box => discrepancy functions with probabilistic guarantees Bounded verification [Sound and relatively complete] Proof rules for sequential composition for unbounded time verification and behavior containment Future: More expressive white boxes, synthesis, monitoring, + = 20 20

21 Links and references Textbook picture links: References : [Fan et al ATVA 15 ] Fan, Chuchu, and Sayan Mitra. "Bounded verification with on-the-fly discrepancy computation." International Symposium on Automated Technology for Verification and Analysis. Springer International Publishing, [Fan 16 et al CAV 16] Fan, Chuchu, et al. "Automatic Reachability Analysis for Nonlinear Hybrid Models with C2E2." International Conference on Computer Aided Verification. Springer International Publishing, [Duggirala et al TACAS 15] Duggirala, Parasara Sridhar, et al. "C2E2: A Verification Tool for Stateflow Models." TACAS [Valiant 84] Valiant, Leslie G. "A theory of the learnable." Communications of the ACM (1984): [Jin et al. HSCC 14] Jin, X., Deshmukh, J. V., Kapinski, J., Ueda, K., & Butts, K. (2014, April). Powertrain control verification benchmark. In Proceedings of the 17th international conference on Hybrid systems: computation and control (pp ). ACM. 21

22 Thank you for your precious time and attention

23 DryVR model semantics Transition graph: Trace: l., t., l /, t /,, l 1 V V Š 5 6 Black-box simulator Trajectory: τ(t) Labeled trajectory set: τ, l TL Hybrid system H = L, Θ, G, TL State: a point in R : L Initial states: Θ L Reach = x, l for some v, t, x, l is reachable from Θ} Reach v: all states reachable in vertex v 23

24 Learning linear separators (cont.) For a subset Γ R R, a linear separator is a pair a, b R / such that x, y Γ, x ay + b ln τ. t τ / t τ. 0 τ / 0, t Γ, ln τ. t τ / (t) τ. 0 τ / (0) γt + ln K Proposition [Valiant 84]: Let ε, δ R p. If k ln then with probability 1 δ, the above ṡ ṫ algorithm finds (a, b) such that err D a, b < ε. err D a, b = D( x, y Γ x > ay + b}) 24

25 Safety verification problem System S requirement R Dry R Bug trace Certificate Is there a behavior of system S violating safety requirement R within time bound T? Yes -> bug-trace -> design improvement No -> safety proof -> certification 25

26 DryVR model semantics + = Transition graph Trace: l., t., l /, t /,, l 1 Black-box simulator Trajectory: τ(t) Labeled trajectory set: τ, l TL Hybrid system H = L, Θ, G, TL State: a point in R : L Reach = x, l for some v, t, x, l is reachable from Θ} Reach v: all states reachable in vertex v 26

27 DryVR s model of lane merge Cruise 1 7 Cruise 5 Accelerate 6 Turn_Right 27

28 DryVR s model of lane merge Accelerate Decelerate Turn_Right Cruise 1 7 Cruise 28