11/18/14 1 ENES 489p Verification and Validation: Logic and Control Synthesis Mumu Xu mumu@umd.edu November 18, 2014 Institute for Systems Research Aerospace Engineering University of Maryland, College Park
11/18/14 2 Table of Contents Verification and Validation Logic Synthesis Reactive Control Synthesis *Slides from EECI2013 Lecture, T. Wongpiromsarn, U. Topcu, R.M. Murray
11/18/14 3 Verification vs. validation Verification: "Are we building the product right" The software should conform to its specification Validation: "Are we building the right product" The software should do what the user really requires V & V must be applied at each stage in the software process Two principal objectives Discovery of defects in a system Assessment of whether the system is usable in an operational situation
11/18/14 4 Basic Concepts Planning for V&V needs to be begin in the early stages of requirements development Fundamental law of faults Failures: externally visible incorrect behavior of a system Error: incorrect internal state Fault: mistake in a system which causes one or more errors and failures FIND AND FIX THE CAUSE OF FAILURES
11/18/14 5 Validation and Verification Plans Two ways to detect and remove defects Consitency checking Simulation Diversity and Redundancy Design Requirement: Weight of the item shall be less than or equal to 134 pounds Verification Requirement: The item weight shall be determined by a scale, the calibration for which is correct, with an accuracy of plus or minus 6 ounces. The item shall be placed on the scale located on a level, stable surface and a reading taken. The measured weight shall be less than 134 pounds and 11 ounces.
11/18/14 6 Verification Traceability Matrices Design Requirement Verification Method Test Analysis Demo Exam Verification Requirement Req 1.1 X Level of Application Req 1.2 X Req 1.3 X
11/18/14 7 Model Checking Process Flow The process flow of model checking Efficient model checking tools automate the process: SPIN, nusmv, TLC,...
11/18/14 8 Temporal Logic Temporal refers to underlying nature of time Linear Branching Two key operators <> eventually property satisfied at some point in future [] always property satisfied now and forever in future Linear Temporal Logic (LTL) Introduced in 1970s (A. Pnueli) Large collection of tools for specification, design, analysis Other temporal logics CTL Computation Tree Logic TCTL Timed CTL MTL Metric Temporal Logic (timed LTL) TLA temporal logic of actions (Leslie Lamport) μ-calculus least fixed point operator (A. Prior, 1950s)
11/18/14 9 Linear Temporal Logic
11/18/14 10 Logic (closed system) Synthesis Closed system: behaviors are generated by the system and not affected by external influences Given: Transition system P LTL formula Compute: A path = of P such that P: composition of two traffic lights 1 ; s0,s0 {g 1 } 1 {g 2 } 2 s1,s0 s0,s1 2 2 1 s1,s1 {g 1,g 2 } = (g 1 g 2 ) g 1 g 2 Sample paths of P: P 1 = (hs 0 s 0 ihs 1 s 0 ihs 1 s 1 ihs 0 s 1 i)! 2 = (hs 0 s 0 ihs 0 s 1 i)! 3 = (hs 0 s 0 ihs 1 s 0 ihs 0 s 0 ihs 0 s 1 i)! 1 2
11/18/14 11 A Controls Interpretation P output y Controller C is a function C : M S! Act C 1 ; s0,s0 {g 1 } 1 {g 2 } 2 s1,s0 s0,s1 2 1 2 1 s1,s1 {g 1,g 2 } s0,s0 1 2 P C 2 s1,s0 s0,s1 = = (hs 0 s 0 ihs 1 s 0 ihs 0 s 0 ihs 0 s 1 i)! = (g 1 g 2 ) g 1 g 2 never(both lights green) [safety] Always eventually light 1 green [liveness] Always eventually light 2 green [liveness] 1 s0,s0 2
11/18/14 12 A Solution Approach Closed system synthesis: non-emptiness of satisfiability problem In synthesis, interesting behaviors are good In verification, interesting behaviors are bad Construct a verification model and claim that Trace(P ) \ Words( )=; Counterexample with negative result is a path that satisfies Positive results means path does not exist
11/18/14 13 Traffic Light System model: P = Specification: ; TS 1 s0: red 1 1 k ; TS 2 s0: red 2 2 s1: green s1: green {g 1 } {g 2 } = (g 1 g 2 ) g 1 g 2 L! (A) =Words( ) q0 (g 1 g 2 ) g 1 g 2 (g 1 g 2 ) g1 g2 (g 1 g 2 ) q1 q2 g 1 g 2 A SPIN code: System model (asynchronous composition): bool g1 = 0, g2 = 0; active proctype TL1() { do :: atomic{ g1 == 0 -> g1 = 1} :: atomic{ g1 == 1 -> g1 = 0 } od } active proctype TL2() { do :: atomic{ g2 == 0 -> g2 = 1} :: atomic{ g2 == 1 -> g2 = 0 } od } Automaton from LTL2BA: never { T0 init: if :: (!g1) (!g2) -> goto T0 init :: (g1 &&!g2) -> goto T1 S1 fi; T1 S1: if :: (!g1) (!g2) -> goto T1 S1 :: (!g1 && g2) -> goto accept S1 fi; accept S1: if :: (!g1) (!g2) -> goto T0 init :: (g1 &&!g2) -> goto T1 S1 fi;
11/18/14 14 Traffic Light System model: P = Specification: ; TS 1 s0: red 1 1 k ; TS 2 s0: red 2 2 s1: green s1: green {g 1 } {g 2 } = (g 1 g 2 ) g 1 g 2 L! (A) =Words( ) q0 (g 1 g 2 ) g 1 g 2 (g 1 g 2 ) g1 g2 (g 1 g 2 ) q1 q2 g 1 g 2 A SPIN code: System model (asynchronous composition): bool g1 = 0, g2 = 0; active proctype TL1() { do :: atomic{ g1 == 0 -> g1 = 1} :: atomic{ g1 == 1 -> g1 = 0 } od } active proctype TL2() { do :: atomic{ g2 == 0 -> g2 = 1} :: atomic{ g2 == 1 -> g2 = 0 } od } s0,s0 s1,s0 s0,s0 =(hs 0 s 0 ihs 1 s 0 ihs 0 s 0 ihs 0 s 1 ihs 0 s 0 ihs 0 s 1 i)! Automaton from LTL2BA: never { T0 init: s0,s1 if s0,s0 s0,s1 :: (!g1) (!g2) -> goto T0 init :: (g1 &&!g2) -> goto T1 S1 fi; T1 S1: if :: (!g1) (!g2) -> goto T1 S1 :: (!g1 && g2) -> goto accept S1 fi; accept S1: if :: (!g1) (!g2) -> goto T0 init :: (g1 &&!g2) -> goto T1 S1 fi;
11/18/14 15 Example: Frog Puzzle (http://www.hellam.net/maths2000/frogs.html) Move all yellow frogs to the right side of pond, and all brown frogs to left side of pond Frogs can only jump in direction they re facing Frogs can either jump one rock forward if the rock is empty or jump over a frog if the next rock has a frog on it and the rock after it is empty
11/18/14 16
11/18/14 17 Logic Synthesis: Frog Puzzle Rock i is not occupied or occupied ri {0, 1} State of frog i: s(f i ) {s 0,s 1...,s 6 } Transition system of frog i: Overall system model: P = F 1 F 2 F 6 r 0 r 2 r 3 r 4 r 5 r 6 r 1 F i s 0 s 1 s 2 s 3 s 4 s 5 s 6 F 1 r 1 s0 r 1 r 2 s1 r 2 s2 r 2 r 3 r 3 r 4 s3 r 4 s4 r 4 r 5 r 5 r 6 s5 r 6 s6 F 2 r 2 s1 s2 r 2 r 3 r 3 r 4 r 4 s3 s4 r 4 r 5 r 5 r 6 r 6 s5 s6 F 3 r 2 r 3 s2 r 3 r 4 s3 r 4 s4 r 4 r 5 r 5 r 6 s5 r 6 s6 = s(f 1 ),s(f 2 ),s(f 3 ) {s 4,s 5,s 6 } s(f 4 ),s(f 5 ),s(f 6 ) {s 0,s 1,s 2 } true q0 p q1 true A p, s(f 1 ),s(f 2 ),s(f 3 ) {s 4,s 5,s 6 } s(f 4 ),s(f 5 ),s(f 6 ) {s 0,s 1,s 2 }
11/18/14 18 Open System Synthesis y P y An open system is a system whose behaviors can be affected by external influence E x E x C CP y Open (synchronous) synthesis: Given a system that describes all the possible actions - plant actions y are controllable - environment actions x are uncontrollable a specification (x, y) find a strategy f(x) for the controllable actions which will maintain the specification against all possible adversary moves, i.e., 8x (x, f(x)) time E x 0 x 1 x 2 x 3 CP y 0 = f(x 0 ) y 1 = f(x 0 x 1 ) y 2 = f(x 0 x 1 x 2 ) y 3 = f(x 0 x 1 x 2 x 3 )
11/18/14 19 Reactive Control Synthesis Reactive systems are open systems that maintain an ongoing interaction with their environment rather than producing an output on termination. Consider the synthesis of a reactive system with input x and output y, specified by the linear temporal formula (x, y). The system contains 2 components S1 (i.e., environment ) and S2 (i.e., reactive module ) - Only S1 can modify x - Only S2 can modify y Want to show that S2 has a winning strategy for y against all possible x scenarios the environment may present to it. - Two-person game: treat environment as adversary S2 does its best, by manipulating y, to maintain (x, y) S1 does its best, by manipulating x, to falsify (x, y) If a winning strategy for S2 exists, we say that (x, y) is realizable x S1 S2 y
11/18/14 20 Runner-Blocker System R B Goal Runner R tries to reach Goal. Blocker B tries to intercept and stop R.
11/18/14 21 Runner-Blocker System lose lose win 7
11/18/14 22 Solving Reactive Control Synthesis Solution given as the winning set Winning set is set of states starting from which there exists a strategy for S 2 to satisfy the specification for all possible behaviors of S 1 A winning strategy can be constructed by saving intermediate values in winning set computation Worst case complexity is double exponential 1 st exponent: Specification to nondeterministic Buchi automaton 2 nd exponent: Covert NBA into deterministic Rabin automaton Similar to closed system synthesis: construct product of system and DRA Find set of states starting from which all possible runs in product automaton are accepting Lower Complexity Cases For specifications of form p, p, p, p controller can be synthesized in O(N 2 ), with N is size of the state space.
11/18/14 23 Game Structures: Runner Blocker s3 Game Structure G =(V, X, Y, e, s, e, s, AP,L,') X := {x}, X = {s 0,s 1,s 2,s 3,s 4 } Y := {y}, Y = {s 0,s 1,s 3,s 4 } e := (x = s 2 ) s0 R B s2 s4 s := (y = s 0 ) e := (x = s 2 ) =) (x 0 6= s 2 ) ^ (x 6= s 2 ) =) (x 0 = s 2 ) s := (y = s 0 _ y = s 4 ) =) (y 0 = s 1 _ y 0 = s 3 ) ^ (y = s 1 _ y = s 3 ) =) (y 0 = s 0 _ y 0 = s 4 ) ^ (y 0 6= x 0 ) s1 ' describes the winning condition, e.g., (y = s 4 )
11/18/14 24 Runner Blocker Example Runner Blocker q3 Play: An infinite sequence = s 0 s 1... of system (blocker + runner) states such that s0 is a valid initial state and (sj, sj+1) satisfies the transition relation of the blocker and the runner q0 R B q2 q4 Strategy: A function that gives the next runner state, given a finite number of previous system states of the current play, the current system state and the next blocker state Winning state: A state starting from which there exists a strategy for the runner to satisfy the winning condition for all the possible behaviors of the blocker q1 Winning game: For any valid initial blocker state sx, there exists a valid initial runner state sy such that (sx, sy) is a winning state Solving game: Identify the set of winning states
11/18/14 25 Solving Game Structures Solving Game Structures General solutions are hard Worst case complexity is double exponential (roughly in number of states) Special cases are easier For a specification of the form p, p, p or p, the controller can be synthesized in O(N 2 ) time where N is the size of the state space Another special case: GR(1) formulas ' =( p 1 ^...^ p m ) =) ( q 1 ^...^ q n ) {z } {z } ' e Thm (Piterman, Sa ar, Pneuli, 2007) A game structure G with a GR(1) winning condition can be solved by a symbolic algorithm in time proportional to nm V 3 ' s More useful form: Can show that this can be converted to GR(1) form