Runtime Verification of Safety/Progress Properties

Transcription

1 untime Verification of Safety/Progress Properties VEIMAG Lab & Université Joseph Fourier Grenoble, France Yliès Falcone and Jean-Claude Fernandez and Laurent Mounier November 24 th 2009, Prague, Czesh epublic (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 1 / 43

2 Verimag DCS Team (Distributed & Complex Systems) (18 permanents, 10 post-doc students, 20 PhD students) specification & design of component-based systems embedded systems service-oriented applications verification and validation techniques model-based techniques program verification test security issues cryptographic protocols software vulnerabilities (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 2 / 43

3 Verification and validation activities within DCS model-based techniques model-checking [IF] model-driven test generation [TGV] simulation, performance evaluation [Glonemo] correct-by-construction paradigm [BIP] program-based techniques static analysis, shape analysis proof [Coq] execution-based techniques test execution [j-post] run-time monitoring, run-time enforcement (j-veto) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 3 / 43

4 Online runtime monitoring [Havelund, osu,... ] A lightweight verification technique, bridging the gap between model-checking and test Given a program P, a (linear) property Π check if P = Π on the current execution run σ of P? synthesize a monitor M Π (= decision procedure) for Π instrument the program to observe relevant events (w.r.t. Π) at each execution step of a run σ of P a verdict is provided by M Π P events σ = Π? Mon Π verdicts (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 4 / 43

5 Example Program P, calling methods e1, e2,... e n Property Π: the 1st method called cannot be called again (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 5 / 43

6 Example Program P, calling methods e1, e2,... e n Property Π: the 1st method called cannot be called again Program (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 5 / 43

7 Example Program P, calling methods e1, e2,... e n Property Π: the 1st method called cannot be called again Program Monitor * e1 * e1 * e2 * * * (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 5 / 43

8 Example Program P, calling methods e1, e2,... e n Property Π: the 1st method called cannot be called again Program Monitor e1 e2 e5 e4 e2 e5 e3 * * e1 * e1 e * * * (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 5 / 43

9 Example Program P, calling methods e1, e2,... e n Property Π: the 1st method called cannot be called again Program Monitor e1 e2 e5 e4 e2 e5 e3 e2 * e1 * e1 * e * * * (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 5 / 43

10 Example Program P, calling methods e1, e2,... e n Property Π: the 1st method called cannot be called again Program Monitor e1 e2 e5 e4 e2 e5 e3 e4e2 * e1 * e1 * e * * * (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 5 / 43

11 Example Program P, calling methods e1, e2,... e n Property Π: the 1st method called cannot be called again Program Monitor e1 e2 e5 * e2e4e2 * e1 * e1 e2 * e5 e4 e3 * * e2 (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 5 / 43

12 untime enforcement [Schneider, Ligatti,... ] modify the current execution sequence when it goes wrong... Monitor M Π acts as a transducer (a filter) for the current run σ by: leaving it unchanged if correct wrt to Π (transparency) changing it into a valid one otherwise (soundness) Program Monitor P o = Π events σ = Π? EM Π memory e1 e2 e5 e4 e2 e5 e3 e2 e4 halt e2e4e2 * e1 * e1 * e2 * halt * e * halt halt (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 6 / 43

13 Main results Some (clear) benefits of runtime verification: well-targeted: what you verify is what you execute scalable: independent w.r.t program size easy to use: do not rely on a program model highly dymanic: set of expected properties can evolve set of program components can evolve Many (potential) application domains: security (OS level, application level), safety-critical systems, pervasive applications, open component-based systems (web services) etc. Several existing tool implementations: JavaMOP, ule, Polymer,... (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 7 / 43

14 But still many open issues... how to instrument the code? quality of the probes, source vs binary instrumentation, etc. how to limit the execution time overhead? monitor integration, efficiency of the decision procedure what properties could be addressed? going beyond safety properties, non-functionnal requirements how to synthesize the corresponding monitors? how to cope with a limited observability/controlability over the program events? etc. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 8 / 43

15 In this talk Focus on V expressivity and monitor synthesis: 1 delineate the space of properties that could be addressed by runtime monitoring & enforcement 2 revisit some previous results on this topic 3 propose an (automated) monitor generation technique 4 present a prototype tool Based on a unified framework: safety-progress property classification [Manna, Pnueli] (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 9 / 43

16 Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] 2 Applicability of untime Validation Techniques 3 untime Verification and Enforcement Monitors 4 j-veto: A Java Verification and Enforcement TOolbox 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 10 / 43

17 Overview Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] Overview The automata view 2 Applicability of untime Validation Techniques 3 untime Verification and Enforcement Monitors 4 j-veto: A Java Verification and Enforcement TOolbox 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 11 / 43

18 Overview Overview (1) General classification of linear temporal properties (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 12 / 43

19 Overview Overview (1) General classification of linear temporal properties Fine-grain definition of classes of properties basic classes: safety, guarantee, response, persistence compound classes: obligation, reactivity (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 12 / 43

20 Overview Overview (1) General classification of linear temporal properties Fine-grain definition of classes of properties basic classes: safety, guarantee, response, persistence compound classes: obligation, reactivity The intuitive/informal idea Safety Guarantee esponse Persistence (all prefixes) ( ψ) (one prefix) ( ψ) (regularly) ( ψ) (continuously) ( ψ) ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ eactivity esponse Persistence Obligation Progress Safety Safety Guarantee (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 12 / 43

21 Overview Examples of properties Example Operating system where an operation op is allowed only when an authorization auth has been granted before. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 13 / 43

22 Overview Examples of properties Example Operating system where an operation op is allowed only when an authorization auth has been granted before. Π 1 : an authorization grant grant auth should precede any occurrence of op is a safety property; (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 13 / 43

23 Overview Examples of properties Example Operating system where an operation op is allowed only when an authorization auth has been granted before. Π 1 : an authorization grant grant auth should precede any occurrence of op is a safety property; Π 2 : the 1st authorization request req auth should be eventually followed by a grant (grant auth) or a deny (deny auth) is a guarantee property; (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 13 / 43

24 Overview Examples of properties Example Operating system where an operation op is allowed only when an authorization auth has been granted before. Π 1 : an authorization grant grant auth should precede any occurrence of op is a safety property; Π 2 : the 1st authorization request req auth should be eventually followed by a grant (grant auth) or a deny (deny auth) is a guarantee property; Π 3 : each occurrence of req auth should be first written in a log file and then answered either with a grant auth or a deny auth without any occurrence of op in the meantime is a response property; (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 13 / 43

25 Overview Examples of properties Example Operating system where an operation op is allowed only when an authorization auth has been granted before. Π 1 : an authorization grant grant auth should precede any occurrence of op is a safety property; Π 2 : the 1st authorization request req auth should be eventually followed by a grant (grant auth) or a deny (deny auth) is a guarantee property; Π 3 : each occurrence of req auth should be first written in a log file and then answered either with a grant auth or a deny auth without any occurrence of op in the meantime is a response property; Π 4 : an incorrect use of operation op should imply that any future call to req auth will always result in a deny auth answer is a persistence property. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 13 / 43

26 Overview Overview (2) Characterization according to several views logical, language-theoretic, topological automata: Streett automata (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 14 / 43

27 Overview Overview (2) Characterization according to several views logical, language-theoretic, topological automata: Streett automata Customizing the SP classification for runtime verification Initially defined for infinite execution sequences Monitoring context Processing incremental finite sequences Verdict taken on finite sequences Our properties: r-properties: (φ, ϕ) φ: the finitary property ϕ: the infinitary property There is be a link between φ and ϕ (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 14 / 43

28 The automata view Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] Overview The automata view 2 Applicability of untime Validation Techniques 3 untime Verification and Enforcement Monitors 4 j-veto: A Java Verification and Enforcement TOolbox 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 15 / 43

29 The automata view Finite state automata: Streett automata Definition of a deterministic Streett m-automaton A tuple (Q, q init, Σ,, {( 1, P 1 ),..., ( m, P m )}) Q is the set of automaton states (q init Q is the initial state), total function : Q Σ Q is the transition function, {( 1, P 1 ),..., ( m, P m )} is the set of accepting pairs, i n, i Q are the sets of recurrent states, and Pi Q are the sets of persistent states. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 16 / 43

30 The automata view Finite state automata: Streett automata Definition of a deterministic Streett m-automaton A tuple (Q, q init, Σ,, {( 1, P 1 ),..., ( m, P m )}) Q is the set of automaton states (q init Q is the initial state), total function : Q Σ Q is the transition function, {( 1, P 1 ),..., ( m, P m )} is the set of accepting pairs, i n, i Q are the sets of recurrent states, and Pi Q are the sets of persistent states. Example (Streett automaton) ack req req Σ Every request is acknowledged, and never two successive requests Σ = {req, ack} ack = {1}, P = (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 16 / 43

31 The automata view Acceptance criteria Acceptance condition for Infinite sequences For σ Σ ω, A accepts σ if i [1, m], vinf (σ, A) i vinf (σ, A) P i where vinf (σ, A): set of states visited infinitely often (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 17 / 43

32 The automata view Acceptance criteria Acceptance condition for Infinite sequences For σ Σ ω, A accepts σ if i [1, m], vinf (σ, A) i vinf (σ, A) P i where vinf (σ, A): set of states visited infinitely often Acceptance condition for Finite sequences For σ Σ s.t. σ = n, A accepts σ if q 1,..., q n Q A, run(σ, A) = q 1 q n q 1 = q init A and i [1, m], q n P i i (This semantics is similar to the semantics of V-LTL [Bauer and al.]) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 17 / 43

33 The automata view Acceptance criteria Acceptance condition for Infinite sequences For σ Σ ω, A accepts σ if i [1, m], vinf (σ, A) i vinf (σ, A) P i where vinf (σ, A): set of states visited infinitely often Acceptance condition for Finite sequences For σ Σ s.t. σ = n, A accepts σ if q 1,..., q n Q A, run(σ, A) = q 1 q n q 1 = q init A and i [1, m], q n P i i (This semantics is similar to the semantics of V-LTL [Bauer and al.]) ack req req Σ Every request is acknowledged, and never two successive requests Σ = {req, ack} ack = {1}, P = (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 17 / 43

34 The automata view The automata view Classification according to syntactic restrictions on automata safety: = and no transition from q P to q P. guarantee: P = and no transition from q to q response: P = persistence: = m-obligation: m-automaton no transition from q Pi to q P i, no transition from q i to q i, m-reactivity: unrestricted m-automaton Safety P P Guarantee esponse Persistence P P esponse Persistence Progress Safety Safety eactivity Obligation Guarantee (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 18 / 43

35 Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] 2 Applicability of untime Validation Techniques Monitorable Properties Enforceable properties 3 untime Verification and Enforcement Monitors 4 j-veto: A Java Verification and Enforcement TOolbox 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 19 / 43

36 Monitorable Properties Classical definition of monitorability [Pnueli,Zaks] Determine verdict of infinite sequences with (finite) observations evaluation depends on the satisfaction of the current sequence and its continuations (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 20 / 43

37 Monitorable Properties Classical definition of monitorability [Pnueli,Zaks] Determine verdict of infinite sequences with (finite) observations evaluation depends on the satisfaction of the current sequence and its continuations Monitorable properties and / -determinacy Considering σ Σ, a r-property Π is said to be: -determined by σ, if Π(σ µ) for all completions µ Σ verdict -determined by σ, if Π(σ µ) for all completions µ Σ verdict σ-monitorable if there exists µ Σ s.t. Π is / -determined by σ µ (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 20 / 43

38 Monitorable Properties Classical definition of monitorability [Pnueli,Zaks] Determine verdict of infinite sequences with (finite) observations evaluation depends on the satisfaction of the current sequence and its continuations Monitorable properties and / -determinacy Considering σ Σ, a r-property Π is said to be: -determined by σ, if Π(σ µ) for all completions µ Σ verdict -determined by σ, if Π(σ µ) for all completions µ Σ verdict σ-monitorable if there exists µ Σ s.t. Π is / -determined by σ µ Truth-domain B 3 = {,,?} determines the class of monitorable properties (MP(B 3 )) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 20 / 43

39 Monitorable Properties Characterization of monitorable properties [V 09] Truth-domain of cardinality 3: Obligation MP({?,, }) Safety and Guarantee properties are monitorable Union and intersection of monitorable properties is monitorable (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 21 / 43

40 Monitorable Properties Characterization of monitorable properties [V 09] Truth-domain of cardinality 3: Obligation MP({?,, }) Safety and Guarantee properties are monitorable Union and intersection of monitorable properties is monitorable Example (Monitorable) ack Σ req req ack = {1}, P = Example (Not monitorable) ack req req 1 2 ack = {1}, P = (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 21 / 43

41 Monitorable Properties Characterization of monitorable properties [V 09] Truth-domain of cardinality 3: Obligation MP({?,, }) Safety and Guarantee properties are monitorable Union and intersection of monitorable properties is monitorable Example (Monitorable) ack Σ req req ack = {1}, P = Non-monitorable properties Example (Not monitorable) ack req req (some) esponse, Persistence, and eactivity properties Impossible to detect or the output sequence of a monitor belongs to (?) 1 2 ack = {1}, P = (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 21 / 43

42 Monitorable Properties Characterization of monitorable properties (cont.) Definition (Characterization of a Streett automaton states: P) Good A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Goodp A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Bad A p = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} Bad A = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} P Good Safety P Goodp Guarantee esponse Bad Bad p Bad Bad p Persistence P P Bad Bad p P Bad Good Good Goodp P Good P Goodp (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 22 / 43

43 Monitorable Properties Characterization of monitorable properties (cont.) Definition (Characterization of a Streett automaton states: P) Good A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Goodp A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Bad A p = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} Bad A = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} P Good Safety P Goodp Guarantee esponse Bad Bad p Bad Bad p Persistence P P Bad Bad p P Bad Good Good Goodp P Good P Goodp Theorem A r-property Π, recognized by A Π, is B 3 -monitorable iff q each(q init ), each(q) ( Bad A Π Good A Π ) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 22 / 43

44 Monitorable Properties Characterization of monitorable properties (cont.) Definition (Characterization of a Streett automaton states: P) Good A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Goodp A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Bad A p = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} Bad A = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} P Good Safety P Goodp Guarantee esponse Bad Bad p Bad Bad p Persistence P Bad P Bad p P Bad Good Good Goodp P Good P Goodp Theorem A r-property Π, recognized by A Π, is B 3 -monitorable iff q each(q init ), each(q) ( Bad A Π Good A Π ) Monitorable ack Σ req req ack (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 22 / 43

45 Monitorable Properties Characterization of monitorable properties (cont.) Definition (Characterization of a Streett automaton states: P) Good A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Goodp A = {q Q A T m i=1 ( i P i ) each A (q) T m i=1 ( i P i )} Bad A p = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} Bad A = {q Q A S m i=1 ( i P i ) each A (q) S m i=1 ( i P i )} P Good Safety P Goodp Guarantee esponse Bad Bad p Bad Bad p Persistence P Bad P Bad p P Bad Good Good Goodp P Good P Goodp Not Monitorable Theorem ack req A r-property Π, recognized by A Π, is B 3 -monitorable iff req q each(q init ), each(q) ( Bad A Π Good A ) Π 1 2 ack (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 22 / 43

46 Monitorable Properties efinement of the notion of monitorability [V 09] Interest of the classical definition: 3-valued truth-domain detection of verdicts for infinitary properties from finite observations (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 23 / 43

47 Monitorable Properties efinement of the notion of monitorability [V 09] Interest of the classical definition: 3-valued truth-domain detection of verdicts for infinitary properties from finite observations Following [Bauer and al.] and the motivations of V-LTL: Is / -determinacy always needed? Answer What happens if the program execution stops here? Distinguish prefixes which evaluated previously to? Considering the truth-domain B 4 = {, p, p, }: p : presumably false p : presumably true (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 23 / 43

48 Monitorable Properties Alternative definition of monitorability Definition (Evaluation of a sequence wrt. a property) [[Π]](σ) = if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = p if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = p if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = if Π(σ) µ Σ Π(σ µ) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 24 / 43

49 Monitorable Properties Alternative definition of monitorability Definition (Evaluation of a sequence wrt. a property) [[Π]](σ) = if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = p if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = p if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = if Π(σ) µ Σ Π(σ µ) Several possible truth-domains: B 2 = {,?} B 3 = {,?, } B 2 = {?, } B 4 = {, p, p, } (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 24 / 43

50 Monitorable Properties Alternative definition of monitorability Definition (Evaluation of a sequence wrt. a property) [[Π]](σ) = if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = p if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = p if Π(σ) µ Σ Π(σ µ) [[Π]](σ) = if Π(σ) µ Σ Π(σ µ) Several possible truth-domains: B 2 = {,?} B 3 = {,?, } B 2 = {?, } B 4 = {, p, p, } Definition (Alternative monitorability) A r-property Π = (φ, ϕ) is said to be monitorable with B iff σ good φ, σ bad φ, [[Π]] B (σ good ) [[Π]] B (σ bad ) We note MP (B), the set of monitorable properties with B. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 24 / 43

51 Monitorable Properties New characterization of monitorable properties Theorem (Multi-valued characterization of monitorability) The sets of monitorable r-properties are: (i) MP (B 2 ) = Safety (ii) MP (B 2 ) = Guarantee (iii) MP (B 3 ) = Safety Guarantee (iv) MP (B 4 ) = eactivity (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 25 / 43

52 Monitorable Properties New characterization of monitorable properties Theorem (Multi-valued characterization of monitorability) The sets of monitorable r-properties are: (i) MP (B 2 ) = Safety (ii) MP (B 2 ) = Guarantee (iii) MP (B 3 ) = Safety Guarantee (iv) MP (B 4 ) = eactivity Example always a Safety property with B 2 = {,?}: a σ bad = a a Σ σ good = a? Σ = {a, a} P = {1} = Σ a 1 2 (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 25 / 43

53 Enforceable properties Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] 2 Applicability of untime Validation Techniques Monitorable Properties Enforceable properties 3 untime Verification and Enforcement Monitors 4 j-veto: A Java Verification and Enforcement TOolbox 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 26 / 43

54 Enforceable properties Enforcement, soundness, and transparency Soundness and transparency: 1 Output sequences are correct: soundness 2 Correct original execution sequences remain unchanged: transparency (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 27 / 43

55 Enforceable properties Enforcement, soundness, and transparency Soundness and transparency: 1 Output sequences are correct: soundness 2 Correct original execution sequences remain unchanged: transparency Our choice: input sequence σ should be modified in a minimal way: σ = Π it should remain unchanged (up to an equivalence relation), σ = Π its longest prefix satisfying Π should be issued. Expected for both finite and infinite execution sequences (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 27 / 43

56 Enforceable properties Enforcement, soundness, and transparency Soundness and transparency: 1 Output sequences are correct: soundness 2 Correct original execution sequences remain unchanged: transparency Our choice: input sequence σ should be modified in a minimal way: σ = Π it should remain unchanged (up to an equivalence relation), σ = Π its longest prefix satisfying Π should be issued. Expected for both finite and infinite execution sequences Consequence: enforceability criterion for Π = (φ, ϕ) Each infinite incorrect sequence has a longest correct prefix, i.e., a finite number of correct prefixes. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 27 / 43

57 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

58 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} a Σ \ {a} 1 2 a Σ \ {a} (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

59 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} Σ \ {a} 1 2 a Infinite incorrect sequence can have an infinite number of correct prefixes: σ bad = (ab) ω Π(σ bad ) since vinf (σ bad, A Π ) = {1, 2} but i N, Π`(ab) i a since P = {1} a Σ \ {a} (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

60 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} Σ \ {a} 1 2 a Infinite incorrect sequence can have an infinite number of correct prefixes: σ bad = (ab) ω Π(σ bad ) since vinf (σ bad, A Π ) = {1, 2} but i N, Π`(ab) i a since P = {1} What is the problem if we build a sound and transparent enforcement monitor? a Σ \ {a} (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

61 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} Σ \ {a} 1 2 a Infinite incorrect sequence can have an infinite number of correct prefixes: σ bad = (ab) ω Π(σ bad ) since vinf (σ bad, A Π ) = {1, 2} but i N, Π`(ab) i a since P = {1} What is the problem if we build a sound and transparent enforcement monitor? IN = a OUT = a a Σ \ {a} (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

62 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} Σ \ {a} 1 2 a Infinite incorrect sequence can have an infinite number of correct prefixes: σ bad = (ab) ω Π(σ bad ) since vinf (σ bad, A Π ) = {1, 2} but i N, Π`(ab) i a since P = {1} What is the problem if we build a sound and transparent enforcement monitor? IN = a b OUT = a a Σ \ {a} (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

63 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} Σ \ {a} 1 2 a Infinite incorrect sequence can have an infinite number of correct prefixes: σ bad = (ab) ω Π(σ bad ) since vinf (σ bad, A Π ) = {1, 2} but i N, Π`(ab) i a since P = {1} What is the problem if we build a sound and transparent enforcement monitor? IN = a b a OUT = a b a a Σ \ {a} (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

64 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} Σ \ {a} 1 2 a Infinite incorrect sequence can have an infinite number of correct prefixes: σ bad = (ab) ω Π(σ bad ) since vinf (σ bad, A Π ) = {1, 2} but i N, Π`(ab) i a since P = {1} What is the problem if we build a sound and transparent enforcement monitor? IN = a b a b OUT = a b a a Σ \ {a} (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

65 Enforceable properties esponse are exactly the enforceable properties Theorem (esponse = EP) Consequence: safety, guarantee, and obligation r-properties are enforceable. Example (Pure persistence property not enforceable) it will be eventually true that a always occur Π = (Σ a +, Σ a ω ) Σ = {a, b}, vinf (σ, A Π ) P and P = {1} Σ \ {a} 1 2 a Infinite incorrect sequence can have an infinite number of correct prefixes: σ bad = (ab) ω Π(σ bad ) since vinf (σ bad, A Π ) = {1, 2} but i N, Π`(ab) i a since P = {1} a Σ \ {a} What is the problem if we build a sound and transparent enforcement monitor? IN = a b a b a OUT = a b a b a an infinite incorrect sequence (e.g. σ bad ) could be produced :-( (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 28 / 43

66 Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] 2 Applicability of untime Validation Techniques 3 untime Verification and Enforcement Monitors Synthesis Composition 4 j-veto: A Java Verification and Enforcement TOolbox 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 29 / 43

67 Synthesis untime verification and enforcement Monitors Definition (Monitor) A is a 5-tuple (Q A, q init A, A, X A, Γ A ) (defined relatively to Σ) a (classical) FSM The set of values X A depends on the purpose of the monitor (verification or enforcement) Γ A : Q A X A, output function, producing values in X A from states. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 30 / 43

68 Synthesis untime verification and enforcement Monitors Definition (Monitor) A is a 5-tuple (Q A, q init A, A, X A, Γ A ) (defined relatively to Σ) a (classical) FSM The set of values X A depends on the purpose of the monitor (verification or enforcement) Γ A : Q A X A, output function, producing values in X A from states. untime Verification and Enforcement Monitors Using P to define the output function Γ A (depends on the current state) For runtime verification: X A = B 4 For runtime enforcement: X A = {halt, store, dump, off } using an internal memory: a FIFO queue (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 30 / 43

69 Synthesis Enforcement operations Enforcement operations: inputs: an event and a memory content (i.e., a sequence of events) outputs: a new memory content and an output sequence Definition (Enforcement operations Ops) Ops 2 ((Σ {ɛ}) Σ ) (Σ Σ ). Ops = {halt, store, dump, off } defined as follows: a Σ {ɛ}, m Σ halt(a, m) = (ɛ, m) store(a, m) = (ɛ, m.a) dump(a, m) = (m.a, ɛ) off (a, m) = (m.a, ɛ) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 31 / 43

70 Synthesis Synthesis from Streett automata [ICISS 08,SAC 09] A Π = (Q A Π, q init A Π, Σ, AΠ, {( 1, P 1 ),..., ( m, P m )}) recognizing Π Correspondence between P and B 4 run ends in Good A Π [[Π]] B4 (σ) =, run ends in Bad A Π run ends in Good A p [[Π]] B4 (σ) = p, Π p [[Π]] B4 (σ) = p, run ends in Bad A Π [[Π]] B4 (σ) =. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 32 / 43

71 Synthesis Synthesis from Streett automata [ICISS 08,SAC 09] A Π = (Q A Π, q init A Π, Σ, AΠ, {( 1, P 1 ),..., ( m, P m )}) recognizing Π Correspondence between P and B 4 run ends in Good A Π [[Π]] B4 (σ) =, run ends in Bad A Π run ends in Good A p [[Π]] B4 (σ) = p, Π p [[Π]] B4 (σ) = p, run ends in Bad A Π [[Π]] B4 (σ) =. Transformation Streett2VM(A Π ) = (Q A Π, qinit A Π, AΠ, B 4, Γ) with Γ : Q A? B 4 s.t. q Q A Π, q Good A Π Γ(q) = q Good A Π p Γ(q) = p q Bad A Π p Γ(q) = p q Bad A Π Γ(q) = Transformation Streett2EM(A Π ) = (Q A Π, qinit A Π, AΠ, Ops, Γ) with Γ : Q A!Π Ops s.t. q Q A Π, q Good A Π Γ(q) = off q Good A Π p Γ(q) = dump q Bad A Π p Γ(q) = store q Bad A Π Γ(q) = halt (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 32 / 43

72 Composition About composition (overview) The idea: order B 4 = {, p, p, } and Ops = {halt, store, dump, off } B 4 : < p< p < Ops : halt < store < dump < off (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 33 / 43

73 Composition About composition (overview) The idea: order B 4 = {, p, p, } and Ops = {halt, store, dump, off } B 4 : < p< p < Ops : halt < store < dump < off How to compose? Build the underlying cartesian product To determine the output (verdict or enforcement operations): Union is taking the max Intersection is taking the min (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 33 / 43

74 Composition About composition (overview) The idea: order B 4 = {, p, p, } and Ops = {halt, store, dump, off } B 4 : < p< p < Ops : halt < store < dump < off How to compose? Build the underlying cartesian product To determine the output (verdict or enforcement operations): Union is taking the max Intersection is taking the min Example (Union of Enforcement Monitors: always a or eventually b ) always a a a 1 Σ 1 2 1: presumably good 2: bad eventually b b b 1 Σ 2 2 1: presumably bad 2: good always a or eventually b (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 33 / 43

75 Composition About composition (overview) The idea: order B 4 = {, p, p, } and Ops = {halt, store, dump, off } B 4 : < p< p < Ops : halt < store < dump < off How to compose? Build the underlying cartesian product To determine the output (verdict or enforcement operations): Union is taking the max Intersection is taking the min Example (Union of Enforcement Monitors: always a or eventually b ) always a a eventually b b dump store always a or eventually b a b Σ 1 halt Σ 2 off (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 33 / 43

76 Composition About composition (overview) The idea: order B 4 = {, p, p, } and Ops = {halt, store, dump, off } B 4 : < p< p < Ops : halt < store < dump < off How to compose? Build the underlying cartesian product To determine the output (verdict or enforcement operations): Union is taking the max Intersection is taking the min Example (Union of Enforcement Monitors: always a or eventually b ) always a a Σ 1 a dump halt eventually b b Σ 2 b store off always a or eventually b a b a a b b a b a b Σ 12 = {ab; ab, ab, ab} a b Σ 12 (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 33 / 43

77 Composition About composition (overview) The idea: order B 4 = {, p, p, } and Ops = {halt, store, dump, off } B 4 : < p< p < Ops : halt < store < dump < off How to compose? Build the underlying cartesian product To determine the output (verdict or enforcement operations): Union is taking the max Intersection is taking the min Example (Union of Enforcement Monitors: always a or eventually b ) always a a Σ 1 a dump halt eventually b b Σ 2 b store off always a or eventually b a b a a b b dump store a b a b Σ 12 = {ab; ab, ab, ab} b off off a Σ 12 (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 33 / 43

78 Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] 2 Applicability of untime Validation Techniques 3 untime Verification and Enforcement Monitors 4 j-veto: A Java Verification and Enforcement TOolbox Overview and property design Monitor Synthesis Monitor Integration 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 34 / 43

79 About j-veto Toolbox for this approach: 2 main stages and additional features Monitor synthesis: Property verification or enforcement Monitor XSLT transformation (XML to XML) Monitor integration: using program-transformation frameworks (here AOP) Additional features: monitor composition (boolean operations), graphic representation of monitors,... (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 35 / 43

80 About j-veto Toolbox for this approach: 2 main stages and additional features Monitor synthesis: Property verification or enforcement Monitor XSLT transformation (XML to XML) Monitor integration: using program-transformation frameworks (here AOP) Additional features: monitor composition (boolean operations), graphic representation of monitors,... Experimental evaluation DaCapo benchmark with properties about the use of Java data structures e.g., Do not update a Collection if an iterator has been created from it Comparison with Java-MOP (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 35 / 43

81 Overview and property design Overview Pattern X {A, E,, P} egular Property ψ(σ a) Monitor Synthesis Automaton like epresentations (Java Objects, XML Automata) Mapping Integration Program Σ a Σ c Directives P Σc Monitor for Π = (X Monitor Integration f (ψ a), X (ψ a)) Program P Σc Automaton Model and Utilities j-veto Graphic epresentations (png, jpg,... ) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 36 / 43

82 Overview and property design Property design Informal Property Program P Σc Property Design Pattern X {A, E,, P} egular Property ψ(σ a) Mapping Σ a Σ c (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 37 / 43

83 Monitor Synthesis Monitor Synthesis Pattern X {A, E,, P} egular Property ψ(σ a) DFA, A ψ DFA2Streett r-property Π = (X f (ψ), X (ψ)) Streett Automaton A Π Streett2Monitor Monitor Synthesis Verification Monitor A?Π Enforcement Monitor A!Π (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 38 / 43

84 Monitor Integration Monitor Integration Original Program P Σc Integration Directives Mapping Σ a Σ c Monitor for Π = (X f (ψ), X (ψ)) Mapping2Aspect MonitorTranslator Event Observer (.aj) Java Monitor AspectJ Compiler Program P Σ c TraceSlicer (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 39 / 43

85 Outline 1 The Safety-Progress Classification of Properties [Manna,Pnueli] 2 Applicability of untime Validation Techniques 3 untime Verification and Enforcement Monitors 4 j-veto: A Java Verification and Enforcement TOolbox 5 Conclusion and future works (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 40 / 43

86 Conclusion Study & implementations of runtime techniques using a general framework (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 41 / 43

87 Conclusion Study & implementations of runtime techniques using a general framework Applicability of untime Techniques Characterization of monitorable properties parameterized according to a truth-domain classical definition (MP(B)) alternative definition (MP (B)) Characterization of enforceable properties (EP) eactivity MP (B 4 ) esponse Persistence EP MP(B 3 ) MP(B 2 ) Obligation MP(B 2 ) Progress MP (B 3 ) Safety Safety Guarantee MP (B 2 ) MP (B 2 ) (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 41 / 43

88 Conclusion Study & implementations of runtime techniques using a general framework Applicability of untime Techniques Characterization of monitorable properties parameterized according to a truth-domain classical definition (MP(B)) alternative definition (MP (B)) Characterization of enforceable properties (EP) eactivity MP (B 4 ) esponse Persistence EP MP(B 3 ) MP(B 2 ) Obligation MP(B 2 ) Progress MP (B 3 ) Safety Safety Guarantee MP (B 2 ) MP (B 2 ) Synthesis procedures to generate runtime and enforcement monitors Prototype tools implementing the aforementioned features (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 41 / 43

89 (some) Future Works Further study the practical feasibility of the approach Observable/Controllable events data dependency between events Memory limitation for the EM How the sets of monitorable and enforceable properties are impacted? Application to specific domains: web service orchestration/choregraphy continuous lifetime verification non-functionnal properties (e.g., power management) etc. (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 42 / 43

90 Some publications Falcone, Y., Fernandez, J-C., Mounier, L., ichier, J-L.: A Test Calculus Framework Applied to Network Policy Testing. In FATES 06. Falcone, Y., Fernandez, J-C., Mounier, L., ichier, J-L.: A Compositional testing Framework Driven by Partial Specifications In TestCOM/FATES 07. Falcone, Y., Fernandez, J-C., Mounier, L., ichier, J-L.: j-post: A Java Toolchain for Property-Oriented Software Testing In MBT 08. Falcone, Y., Fernandez, J.C., Mounier, L.: Enforcement Monitoring wrt. the Safety-Progress Classification of Properties. In SAC 09. Falcone, Y., Fernandez, J.C, Mounier, L.: untime Verification of Safety-Progress Properties. In V 09. Falcone, Y., Mounier, L., Fernandez, J-C., ichier, J-L.: untime Enforcement Monitors: composition, synthesis, and enforcement abilities under review at FMSD Thank you for your attention (L. Mounier - Verimag/UJF) untime Verif. of SP properties 24/11/09 - Prague 43 / 43