Software Specification 2IX20

Transcription

1 Software Specification 2IX20 Julien Schmaltz (with slides jointly with J. Tretmans, TNO&RUN) Lecture 13: Model-Based Testing III (real-timed systems)

2 Correctness Implementation Relation ioco i ioco s = def σ Straces (s) : out (i after σ) out (s after σ) Intuition: i ioco-conforms to s, iff if i produces output x after trace σ, then s can produce x after σ if i cannot produce any output after trace σ, then s cannot produce any output after σ ( quiescence δ )

3 Correctness Implementation Relation ioco i ioco s = def σ Straces (s) : out (i after σ) out (s after σ) δ ( p ) = L U {τ}. p Straces ( s ) = { σ ( L { δ } )* s σ } out ( p ) = { L U p } { δ δ ( p ) } out ( P ) = { out ( p ) p P } p after σ = { p p σ p }

4 Implementation Relation ioco 2 2 ioco ioco!tea!choc 2 δ!tea s? but2 2 ioco ioco!tea!choc

5 ?a?a?b τ δ?b?b!y!y (i) (s) Is (i) ioco-conforming to (s)? out (?a δ?b) =?

6 Timed Model-Based Testing!Aim: understand more about real-time implementation relations which ones are useful and intuitive compare relations from the literature find constraints under which different relations are equal allow tool comparison: Uppaal-TRON, TTG, Timed-TorX!Challenges: Is time input or output? Quiescence: How long is there never eventually no output?

7 Timed Model-Based Testing!In many systems real-time properties are crucial!approach: Extension of IOLTS/ioco theory Timed Input Output Transition Systems (TIOTS) Timed Implementation Relations: build on ioco Note that there are also approaches for timed FSM

8 Timed Model-Based Testing!Literature (only for transition systems): Krichen, Tripakis: tiocogre Nielsen, Mikucionis, Skou, Larsen: rtioco Brandán Briones, Brinksma: tiocom Bohnenkamp, Belinfante (Timed TorX) Khoumsi, Jéron, Marchand Schmaltz, Tretmans: tioco, tioco η,tioco ζ

9 Timed Automata: Examples [c <= 5] the system may produce within 5 time units, or no output is ever produced [c >= 5] [c<=5] the system must produce within 5 time units the system may produce (after 5 time units) at any time, or no output is ever produced

10 Simple testing questions Is (i) a valid implementation of (s)? (i) (s)!y How long do we need to reject (i)? (i) (s)

11 UPPAAL-TRON

12 Conformance relation: relativized conformance!developed by Nielsen, Mikucionis, Skou, Larsen: Conformance relation: rtioco e relative to an environment!based on UPPAAL s engine for Timed Automata Timed traces?a 2 3 0,4?b!y ttraces ( s ) = { σ ( L R 0 )* s σ } Output set µ out aa (q) = { µ L U R 0 q } Confto i rtioco e s = def σ ttraces (e) : out aa (i e after t σ) out aa (s e after t σ)

13 Output set of rtioco e [c<=b] [c <= b]?a?a τ?a τ [c < M] R 0 { } R 0 { } R 0 { } [0:b] R 0 R 0 R 0 µ out aa (q) = { µ L U R 0 q }

14 relativized conformance!issues to compose with the environment the specification must be input-enabled conformance is only relative to an environment that has to be defined!solution Variation of rtioco e defined in a way similar to ioco i tioco aa s = def σ ttraces (s) : out aa (i after t σ) out aa (s after t σ) Proposition. Let i,s be two input-enabled TIOTS(L I,L U ). Let e be an input-enabled TIOTS(L U,L I ). Then, we have (i e) tioco aa (s e) iff i rtioco e s

15 Is (i) a valid implementation of (s)? (i) (s) i tioco aa s = def σ ttraces (s) : out aa (i after t σ) out aa (s after t σ) µ out aa (q) = { µ L U R 0 q }

16 ?a?a?b τ?b?b!y!y (i) (s) i tioco aa s = def σ ttraces (s) : out aa (i after t σ) out aa (s after t σ) µ out aa (q) = { µ L U R 0 q }

17 tioco: Examples c:=0 c<=7 tioco tioco c:=0 c<=9 c==7 c>=3 c:=0 c<10 c>=5 c:=0 c<9 c>=7 tioco tioco c:=0 true

18 Test Generation Algorithm Z := {(s 0, e 0 )} while Z #iterations T do switch(action, delay, restart) randomly: action: // offer an input if EnvOutput(Z) // there is a possible input randomly choose i EnvOutput(Z) send i to IUT, Z := Z after t i delay: // wait for an output randomly choose d Delays(Z) sleep for d time units or wake up on output o at d d if o occurs then Z := Z after t d if o ImpOutput(Z) then return fail else Z:= Z after t o else Z := Z after t d // no output during delay d restart: Z := {(s 0, e 0 )}, reset IUT If Z = then return fail else return pass

19 Test Generation: Examples (1) Some test runs: c:=0 c:=0 but 7 coffee pass c<10 c<=7 but 3 4 coffee pass c>=5 c==7 100 but 7 coffee pass (s) (i 1 ) action: // offer an input if EnvOutput(Z) // there is a possible input randomly choose i EnvOutput(Z) send i to IUT, Z := Z after t i!but (e)?coffee delay: // wait for an output randomly choose d Delays(Z) sleep for d time units or wake up on output o at d d if o occurs then Z := Z after t d if o ImpOutput(Z) then return fail else Z:= Z after t o else Z := Z after t d // no output during delay d

20 Test Generation: Examples (2) Some test runs: c:=0 c:=0 but 7 coffee pass c<10 c<=9 but 9 coffee pass c>=5 c>=3 but 3 coffee fail (s) (i 2 ) action: // offer an input if EnvOutput(Z) // there is a possible input randomly choose i EnvOutput(Z) send i to IUT, Z := Z after t i!but (e)?coffee delay: // wait for an output randomly choose d Delays(Z) sleep for d time units or wake up on output o at d d if o occurs then Z := Z after t d if o ImpOutput(Z) then return fail else Z:= Z after t o else Z := Z after t d // no output during delay d

21 Test Generation: Completeness!The test generation algorithm is sound w.r.t to rtioco e!the test generation algorithm is exhaustive for rtioco e in the limit, i.e., when T if not (i rtioco e s) then the algorithm detects the error with probability 1!Assumptions on the IUT IUT is a non-blocking input-enabled TIOTS IUT is deterministic IUT has isolated outputs

22 Simple testing questions (i) (s) Is (i) a valid implementation of (s)? (i)!y (s) How long do we need to reject (i)?... very long...

23 UPPAAL-TRON!Implemented as an extension UPPAAL Relativized conformance On-the-fly test generation Application to (few) case-studies Seems feasible in practice!issues Strong test hypothesis Infinite output set Completeness if tests run an infinite amount of time Backward compatibility with ioco!related work Off-line generation using timed game (UPPAAL-TIGA) TTG tool (on-the-fly) by Krichen and Tripakis Timed-TorX