Model-Based Testing: Testing from Finite State Machines

Transcription

1 Model-Based Testing: Testing from Finite State Machines Mohammad Mousavi University of Leicester, UK IPM Summer School 2017 Mousavi FSM-Based Testing IPM / 64

2 Finite State Machines Outline 1 Finite State Machines 2 Final State Identification 3 Initial State Identification 4 Conformance Testing 5 Machine Identification 6 Reversibility and Testing Mousavi FSM-Based Testing IPM / 64

3 Finite State Machines Context: FSM-Based Conformance Testing Problem Given an FSM spec M and a black-box implementation I, does I conform to M, i.e., does I implement the same output and transfer function as M? Mousavi FSM-Based Testing IPM / 64

4 Finite State Machines Context: Testing Finite-State Machines Problem Given an FSM spec M and a black-box implementation I, does I conform to M, i.e., does I implement the same output and transfer function as M? Solution: Complete Test Suite For each state s and input i in the spec: 1 bring the FSM to s, 2 apply i and observe the expected output o, and 3 check the target state s Mousavi FSM-Based Testing IPM / 64

5 Finite State Machines Context: Testing Finite-State Machines Problem Given an FSM spec M and a black-box implementation I, does I conform to M, i.e., does I implement the same output and transfer function as M? Solution: Complete Test Suite For each state s and input i in the spec: 1 bring the FSM to s, (final state identification) 2 apply i and observe the expected output o, and 3 check the target state s (initial state identification / verification ) Mousavi FSM-Based Testing IPM / 64

6 Finite State Machines Notations I ɛ : empty sequence next : target state after a sequence of inputs (naturally generalized to sets of states:next(s, x)) by definition:next(s, ɛ) = s a/1 b/1 s 1 s 2 b/1 s 3 Mousavi FSM-Based Testing IPM / 64

7 Finite State Machines Notations I ɛ : empty sequence next : target state after a sequence of inputs (naturally generalized to sets of states:next(s, x)) by definition:next(s, ɛ) = s next(s 1, bba) = next(next(s 1, b), ba) = next(s 2, ba) = next(next(s 2, b), a) = next(s 3, a) = s 3 a/1 b/1 b/1 s 1 s 2 s 3 Mousavi FSM-Based Testing IPM / 64

8 Finite State Machines Notations II output : action sequence after a sequence of inputs (generalized to sets of states:output(s, x)) assumption: output(s, ɛ) = ɛ b/1 s 1 a/1 s 2 b/1 s 3 Mousavi FSM-Based Testing IPM / 64

9 Finite State Machines Notations II output : action sequence after a sequence of inputs (generalized to sets of states:output(s, x)) assumption: output(s, ɛ) = ɛ output(s 1, bba) = output(s 1, b) output(next(s 1, b), ba) = 1 output(s 2, ba) = 1 output(s 2, b) output(next(s 2, b), a) = 1 1 output(s 3, a) = a/1 b/1 b/1 s 1 s 2 s 3 Mousavi FSM-Based Testing IPM / 64

10 Final State Identification Outline 1 Finite State Machines 2 Final State Identification 3 Initial State Identification 4 Conformance Testing 5 Machine Identification 6 Reversibility and Testing Mousavi FSM-Based Testing IPM / 64

11 Final State Identification Final State Identification Problem: Given an FSM, we do not know which state it is in Solution: Perform a test (homing sequence), observe output sequence and determine the final state of the machine An FSM is reduced when each two different states can show different output on at least one sequence of inputs Mousavi FSM-Based Testing IPM / 64

12 Final State Identification Final State Identification Problem: Given an FSM, we do not know which state it is in Solution: Perform a test (homing sequence), observe output sequence and determine the final state of the machine An FSM is reduced when each two different states can show different output on at least one sequence of inputs Reduced FSM always has a homing sequence FSM that is not reduced may not have a homing sequence Mousavi FSM-Based Testing IPM / 64

13 Final State Identification Determining a homing sequence Algorithm: (for reduced machine) 1 let x = ɛ, Mousavi FSM-Based Testing IPM / 64

14 Final State Identification Determining a homing sequence Algorithm: (for reduced machine) 1 let x = ɛ, 2 while there exists a set B of states such that next(b, x) > 1 and for each two s, s B, output(s, x) = output(s, x), 1 take two states s, s next(b, x) (with s s ) 2 find a sequence y, separating s and s i.e., output(s, y) output(s, y) 3 x := x y 4 partition B into the subsets with the same output after x Mousavi FSM-Based Testing IPM / 64

15 Final State Identification Example s 1 b/1 s 2 b/1 s 3 a/1 Partition of S: {{s 1, s 2, s 3 }} Take B = {s 1, s 2, s 3 }, next(b, ɛ) = B. Take: s 1 and s 2 Mousavi FSM-Based Testing IPM / 64

16 Final State Identification Example s 1 b/1 s 2 b/1 s 3 a/1 Partition of S: {{s 1, s 2, s 3 }} Take B = {s 1, s 2, s 3 }, next(b, ɛ) = B. Take: s 1 and s 2 Separating sequence: a Output sequences: output(s 1, a) = output(s 3, a) = 0 and output(s 2, a) = 1 New partition: {{s 1, s 3 }, {s 2 }} Mousavi FSM-Based Testing IPM / 64

17 Final State Identification Example s 1 b/1 s 2 b/1 s 3 a/1 Partition of S: {{s 1, s 3 }, {s 2 }} Take B = {s 1, s 3 }, next(b, a) = B. Take: s 1 and s 3. Separating sequence: b Output sequences: output(s 1, b) = 1 and output(s 3, b) = 0 New partition: {{s 1 }, {s 3 }, {s 2 }} Mousavi FSM-Based Testing IPM / 64

18 Final State Identification Example s 1 b/1 s 2 b/1 s 3 a/1 Partition of S: {{s 1 }, {s 3 }, {s 2 }} Homing sequence: a b Mousavi FSM-Based Testing IPM / 64

19 Initial State Identification Outline 1 Finite State Machines 2 Final State Identification 3 Initial State Identification 4 Conformance Testing 5 Machine Identification 6 Reversibility and Testing Mousavi FSM-Based Testing IPM / 64

20 State identification Initial State Identification Problem: Given an FSM, can we determine the initial state of the FSM? An input sequence that solves this problem is a distinguishing sequence. Mousavi FSM-Based Testing IPM / 64

21 Initial State Identification Distinguishing sequence A distinguishing sequence for a machine is a rooted tree T with exactly S leaves such that internal nodes are labeled with input symbols leaves are labeled with states edges are labeled with output symbols for each node, the labels of the outgoing edges are different for each leaf, if x and y are input and output sequence on path from root to the leaf and the leaf is labeled by state s, then output(s, x) = y Mousavi FSM-Based Testing IPM / 64

22 Initial State Identification Example a/1 s 1 s 6 s 2 a/1 s 5 s 3 a/1 s 4 Mousavi FSM-Based Testing IPM / 64

23 Initial State Identification Example a/1 s 1 a 1 a 0 b 1 0 s 6 s 5 a/1 s 4 s 2 s 3 a/1 b a 1 0 s 6 0 b a 0 0 b 1 s 5 a 0 0 s 2 1 s 4 a 0 s 1 1 s 3 Mousavi FSM-Based Testing IPM / 64

24 Initial State Identification Non-existence of distinguishing sequence b/1 s 1 s 2 b/1 s 3 a/1 distinguishing sequence cannot start with a because then s 1 and s 2 are not distinguishable since from both s 1 is reached with output 0 distinguishing sequence cannot start with b because then s 2 and s 3 are not distinguishable since from both s 2 is reached with output 1 Mousavi FSM-Based Testing IPM / 64

25 Initial State Identification Existence of distinguishing sequence An input a is valid for a set C of states if it does not merge any two states s and s from C without distinguishing them, i.e., s,s C output(s, a) output(s, a) next(s, a) next(s, a) a/1 s 1 s 6 s 5 s 2 s 3 a/1 Input symbol b is not valid for set of states S Input symbol a is valid for set of states S a/1 s 4 Mousavi FSM-Based Testing IPM / 64

26 Initial State Identification Algorithm 1 Start with partition π of S with only one block {S}. 2 While there is a block B π with B > 1, 1 Take a valid input symbol a I for B such that two states s, s B (s s ), output(s, a) output(s, a) or move to states in different blocks of π, 2 refine the partition π by replacing block B by a set of new blocks, where two states in B are assigned to the same block in the new partition iff they produce the same output on a and move to the same block in π. Property An FSM has a distinguishing sequence iff the final partition contains singleton sets. Mousavi FSM-Based Testing IPM / 64

27 Initial State Identification Example a/1 s 1 s 6 s 2 Initial partition π = {S} Input symbol b is not valid s 5 s 3 a/1 Input symbol a is valid New partition: π = {{s 1, s 3, s 5 }, {s 2, s 4, s 6 }} a/1 s 4 Mousavi FSM-Based Testing IPM / 64

28 Initial State Identification Example a/1 s 6 s 5 s 1 s 2 s 3 a/1 Initial partition π = {{s 1, s 3, s 5 }, {s 2, s 4, s 6 }} Input symbol b is valid for {s 1, s 3, s 5 } New partition: π = {{s 1 }, {s 3, s 5 }, {s 2, s 4, s 6 }} a/1 s 4 Mousavi FSM-Based Testing IPM / 64

29 Initial State Identification Example a/1 s 6 s 5 s 1 s 2 s 3 a/1 Initial partition π = {{s 1 }, {s 3, s 5 }, {s 2, s 4, s 6 }} Input symbol a is valid for {s 2, s 4, s 6 } New partition: π = {{s 1 }, {s 3, s 5 }, {s 2, s 4 }, {s 6 }} a/1 s 4 Mousavi FSM-Based Testing IPM / 64

30 Initial State Identification Example a/1 s 6 s 5 s 1 s 2 s 3 a/1 Initial partition π = {{s 1 }, {s 3, s 5 }, {s 2, s 4 }, {s 6 }} Input symbol b is valid for {s 3, s 5 } New partition: π = {{s 1 }, {s 3 }, {s 5 }, {s 2, s 4 }, {s 6 }} a/1 s 4 Mousavi FSM-Based Testing IPM / 64

31 Initial State Identification Example s 6 s 5 a/1 s 1 Initial partition π = {{s 1 }, {s 3 }, {s 5 }, {s 2, s 4 }, {s 6 }} a/1 s 4 s 2 s 3 a/1 Input symbol a is valid for {s 2, s 4 } New partition: π = {{s 1 }, {s 3 }, {s 5 }, {s 2 }, {s 4 }, {s 6 }} Thus FSM has a distinguishing sequence Mousavi FSM-Based Testing IPM / 64

32 Initial State Identification Example a/1 s 1 s 6 s 2 a/1 Initial partition π = {{s 1 }, {s 3 }, {s 5 }, {s 2 }, {s 4 }, {s 6 }} Thus FSM has a distinguishing sequence s 5 s 3 a/1 s 4 Mousavi FSM-Based Testing IPM / 64

33 Initial State Identification Finite State Machines (recap) states: abstractions of the past, transitions: reactions to input by producing output and moving to a new state state transition function next: S I S output function output: S I O s 0 t/1 t/0 s 1 Mousavi FSM-Based Testing IPM / 64

34 Initial State Identification Finite State Machines (recap) states: abstractions of the past, transitions: reactions to input by producing output and moving to a new state state transition function next: S I S output function output: S I O next and output are generalized to sets of states and sequences of inputs. s 0 t/1 t/0 s 1 Mousavi FSM-Based Testing IPM / 64

35 Initial State Identification Finite State Machines (recap) states: abstractions of the past, transitions: reactions to input by producing output and moving to a new state state transition function next: S I S output function output: S I O next and output are generalized to sets of states and sequences of inputs. Assumption: FSM s are reduced, deterministic and complete. s 0 t/1 t/0 s 1 Mousavi FSM-Based Testing IPM / 64

36 Essential Notions Initial State Identification 1 Reduced FSM: for all distinct s, s S, there exists an x I such that output(s, x) output(s, x). I.e., x separates s and s. 2 Completeness: For each input a, and state s, there exists a s such that next(s, a) = s. 3 Determinism: For each input a, and state s, there exists at most one s such that next(s, a) = s. Mousavi FSM-Based Testing IPM / 64

37 Initial State Identification Equivalence 1 State equivalence: for s S and s S s s x I output(s, x) = output(s, x) 2 Machine equivalence M M s S s S s s s S s S s s Mousavi FSM-Based Testing IPM / 64

38 Conformance Testing Outline 1 Finite State Machines 2 Final State Identification 3 Initial State Identification 4 Conformance Testing 5 Machine Identification 6 Reversibility and Testing Mousavi FSM-Based Testing IPM / 64

39 Conformance Testing Five Fundamental Testing Problems 1 Determine the final state homing sequence: a testcase to reveal the final state Mousavi FSM-Based Testing IPM / 64

40 Conformance Testing Five Fundamental Testing Problems 1 Determine the final state homing sequence: a testcase to reveal the final state 2 State identification (identify the initial state) adaptive distinguishing sequence Mousavi FSM-Based Testing IPM / 64

41 Conformance Testing Five Fundamental Testing Problems 1 Determine the final state homing sequence: a testcase to reveal the final state 2 State identification (identify the initial state) adaptive distinguishing sequence 3 Conformance testing (is blackbox A equivalent to the FSM?) Mousavi FSM-Based Testing IPM / 64

42 Conformance Testing Five Fundamental Testing Problems 1 Determine the final state homing sequence: a testcase to reveal the final state 2 State identification (identify the initial state) adaptive distinguishing sequence 3 Conformance testing (is blackbox A equivalent to the FSM?) 4 Machine identification (derive the FSM from a blackbox) Mousavi FSM-Based Testing IPM / 64

43 Conformance Testing Basic Idea Specification: FSM A Implementation: A blackbox B, only input-output observable Problem: given the specification determine a testcase (a sequence of inputs) to determine whether A B (Also called: fault detection, machine testing) Mousavi FSM-Based Testing IPM / 64

44 Conformance Testing Basic Idea Specification: FSM A Implementation: A blackbox B, only input-output observable Problem: given the specification determine a testcase (a sequence of inputs) to determine whether A B (Also called: fault detection, machine testing) Challenge: for each testcase t, B can always behave the same as A up to t Mousavi FSM-Based Testing IPM / 64

45 Conformance Testing Example specification: s 0 Mousavi FSM-Based Testing IPM / 64

46 Conformance Testing Example specification: s 0 suppose that sequence a n is the testcase (the answer to the conformance testing problem) Mousavi FSM-Based Testing IPM / 64

47 Conformance Testing Example specification: s 0 suppose that sequence a n is the testcase (the answer to the conformance testing problem) for any n, some incorrect implementation may pass the test: n s 0 s n s n a/1 Mousavi FSM-Based Testing IPM / 64

48 Conformance Testing Simplifying Assumptions Assumptions on A strongly connected: (testcase long enough each state visited) reduced: (equivalence: interesting / efficient on reduced FSMs) s 0 t/1 t/0 s 1 Mousavi FSM-Based Testing IPM / 64

49 Conformance Testing Simplifying Assumptions Assumptions on A strongly connected: (testcase long enough each state visited) reduced: (equivalence: interesting / efficient on reduced FSMs) A has the following transitions set(j)/0: a transition from each state to state j, status/i: a self-loop indicating the current state set(0) /0 set(0)/0 t/0 status/1 s 0 s 1 t/1 status/0 set(1) /0 set(1)/0 Last assumption: only for convenience; to be dropped later. Mousavi FSM-Based Testing IPM / 64

50 Conformance Testing Simplifying Assumptions Assumptions on B constant FSM (should not change; should be finite) at most S A states (an upper bound is needed; here, only transfer and output faults tested no new states due to faults) Mousavi FSM-Based Testing IPM / 64

51 Conformance Testing Fault Model set(0)/0 status/0 set(0)/0 status/0 set(0)/0 status/0 s 0 s 0 s 0 set(0) /0 t/0 t/1 set(1) /0 set(0) /0 t/0 t/1 set(1) /0 status/1 set(0) /0 t/1 t/1 set(1) /0 s 1 s 1 s 1 status/1 set(1)/0 set(1)/0 status/1 set(1)/0 Specification Transfer Fault Output Fault Mousavi FSM-Based Testing IPM / 64

52 Conformance Testing Basic Algorithm For each state s and input a in A: set the state to s, supply input a and check in B whether 1 output is output(s, a), and 2 the target is next(s, a). N.B. all transitions are covered by the algorithm (a transition tour is taken). Mousavi FSM-Based Testing IPM / 64

53 Conformance Testing Basic Algorithm For each state s and input a in A: set the state to s, using set(s) supply input a and check in B whether 1 output is output(s, a) (observe the output), and 2 the target is next(s, a) (supply status and observe the output). N.B. all transitions are covered by the algorithm (a transition tour is taken). Mousavi FSM-Based Testing IPM / 64

54 Conformance Testing Example testcase: set(0), status, status, status, rst, status, t, status, set(1), status, status, status, t, status, set(1), rst, status, set(1), set(0), status set(0)/0 status/0 s 0 set(0) /0 t/0 t/1 set(1) /0 s 1 status/1 set(1)/0 Specification 0,0, 0,0, 0, 0, 1, 1, 0,1, 1,1, 0, 0, 0,0,0, 0,0,0 Mousavi FSM-Based Testing IPM / 64

55 Conformance Testing Example testcase: set(0), status, status, status, rst, status, t, status, set(1), status, status, status, t, status, set(1), rst, status, set(1), set(0), status set(0)/0 status/0 set(0) /0 t/0 s 0 t/1 set(1) /0 set(0)/0 set(0) /0 t/0 s 0 t/1 status/0 set(1) /0 set(0)/0 s 0 status/1 set(0) t/1 /0 t/1 status/0 set(1) /0 s 1 s 1 s 1 status/1 set(1)/0 set(1)/0 status/1 Specification Transfer Fault Output Fault 0,0, 0,0, 0, 0, 1, 1, 0,1, 1,1, 0, 0, 0,0,0, 0,0,0 set(1)/0 Mousavi FSM-Based Testing IPM / 64

56 Conformance Testing Example testcase: set(0), status, status, status, rst, status, t, status, set(1), status, status, status, t, status, set(1), rst, status, set(1), set(0), status set(0)/0 status/0 set(0) /0 t/0 s 0 t/1 set(1) /0 set(0)/0 set(0) /0 t/0 s 0 t/1 status/0 set(1) /0 set(0)/0 s 0 status/1 set(0) t/1 /0 t/1 status/0 set(1) /0 s 1 s 1 s 1 status/1 set(1)/0 set(1)/0 status/1 Specification Transfer Fault Output Fault 0,0, 0,0, 0, 0, 1, 1, 0,1, 0,0, 0, 0, 0, 0, 1,1, 0,1, set(1)/0 1,1, 0, 0, 0,0,0, 0,0,0 1, 0, 1,1, 0,0,0 0,0,0 Mousavi FSM-Based Testing IPM / 64

57 Conformance Testing Example testcase: set(0), status, status, status, rst, status, t, status, set(1), status, status, status, t, status, set(1), rst, status, set(1), set(0), status set(0)/0 status/0 set(0) /0 t/0 s 0 t/1 set(1) /0 set(0)/0 set(0) /0 t/0 s 0 t/1 status/0 set(1) /0 set(0)/0 s 0 status/1 set(0) t/1 /0 t/1 status/0 set(1) /0 s 1 s 1 s 1 status/1 set(1)/0 set(1)/0 status/1 set(1)/0 Specification Transfer Fault Output Fault 0,0, 0,0, 0, 0, 1, 1, 0,1, 0,0, 0, 0, 0, 0, 1,1, 0,1,0,0, 0, 0, 0, 0, 1,1, 0,1, 1,1, 0, 0, 0,0,0, 0,0,0 1, 0, 1,1, 0,0,0 0,0,0 1,1, 1, 0, 0,0,0 0,0,0 Mousavi FSM-Based Testing IPM / 64

58 Conformance Testing Transfer Sequence FSM s usually have no set(j): use homing sequence and transfer sequences A transfer sequence τ(i, j): next(s i, τ(s i, s j )) = s j τ(i, j) need not be unique; the shortest path can be found efficiently Examples: HS = rst τ(0, 1) = t, τ(1, 0) = t or rst. s 0 t/1 t/0 status/1 s 1 status/0 Mousavi FSM-Based Testing IPM / 64

59 Conformance Testing Algorithm with Transfer/Homing Sequences 1 Go to a known final state s 2 For each state s and input a in A: set the state from s (the current known state) to s, supply input a and check in B whether 1 output is output(s, a), and 2 the target is next(s, a) Mousavi FSM-Based Testing IPM / 64

60 Conformance Testing Algorithm with Transfer/Homing Sequences 1 Go to a known final state s (using the homing sequence) 2 For each state s and input a in A: set the state from s (the current known state) to s, using τ(s, s) supply input a and check in B whether 1 output is output(s, a) (observe the output), and 2 the target is next(s, a) (supply status and observe the output, if successful let s = next(s, a)) Mousavi FSM-Based Testing IPM / 64

61 Conformance Testing Example testcase: first supply HS = rst, then supply ts, where: ts = status, status, rst, status, t, status, status, status, rst, status, t, t, status t/0 status/0 s 0 t/1 t/0 s 0 t/1 status/0 status/1 t/1 s 0 t/1 status/0 s 1 s 1 status/1 s 1 status/1 Specification Transfer Fault Output Fault 0, 0,0, 0,0, 1,1, 0, 0,0, 0, 0, 1,1, 0, 0,0, 0,0, 1,1, 1,1, 0,0, 1,0,0 1,0,, 0,0, 1,0,0 1,1, 0,0, 1,1,0 Mousavi FSM-Based Testing IPM / 64

62 Status, Realistic? Conformance Testing Sometimes present: registers in hardware, state dumps (logs) in protocols Usually not: let s do without them and apply state identification (e.g., preset or adaptive distinguishing sequence) Challenge: distinguishing sequences change the state Mousavi FSM-Based Testing IPM / 64

63 Conformance Testing Algorithm with Nuts and Bolts 1 Go to a known final state s 0 (homing sequence) Mousavi FSM-Based Testing IPM / 64

64 Conformance Testing Algorithm with Nuts and Bolts 1 Go to a known final state s 0 (homing sequence) 2 Take a sequence of all states s 0,..., s n+1, where s 0 = s n+1. (state tour or state cover seq.) and let i = 0: 1 supply the DS and check if FSM is in s i, 2 FSM is in t i (due to DS), supply τ(t i, s i+1 ) and repeat for i := i + 1, until i = n + 1 Mousavi FSM-Based Testing IPM / 64

65 Conformance Testing Algorithm with Nuts and Bolts 1 Go to a known final state s 0 (homing sequence) 2 Take a sequence of all states s 0,..., s n+1, where s 0 = s n+1. (state tour or state cover seq.) and let i = 0: 1 supply the DS and check if FSM is in s i, 2 FSM is in t i (due to DS), supply τ(t i, s i+1 ) and repeat for i := i + 1, until i = n For each state s i and input label a, (assuming that current state is s j ) 1 supply τ(s, j s i 1), DS, τ(t i 1, s i ), take yourself back to the safe path, 2 supply the input a, DS and check whether the output is output(s i, a) and FSM is in next(s i, a), the current state is s (due to a, DS) j+1 N.B. In step 3.1, one may apply τ(t i 1, s i ), if s j = t i 1 and skip the step if s j = s i. Mousavi FSM-Based Testing IPM / 64

66 Conformance Testing Example s 0 s 0 t/0 t/1 t/1 t/1 s 1 s 1 Specification Output Fault Mousavi FSM-Based Testing IPM / 64

67 Conformance Testing Example testcase: first supply HS = rst, output = 0 (no other choice!), then supply the state tour: t, t, then start testing t, t, rst, t, t, t, rst, t s 0 s 0 t/0 t/1 t/1 t/1 s 1 s 1 Specification Output Fault 0, 1, 0, 1, 0, 0, 1, 0,1 0, 1 Mousavi FSM-Based Testing IPM / 64

68 Conformance Testing Example testcase: first supply HS = rst, output = 0 (no other choice!), then supply the state tour: t, t, then start testing t, t, rst, t, t, t, rst, t s 0 s 0 t/0 t/1 t/1 t/1 s 1 s 1 Specification Output Fault 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0,1 0, 1 1, 1, 0,1, 1, 1, 0, 1 Mousavi FSM-Based Testing IPM / 64

69 Conformance Testing Example s 0 s 0 b/1 b/1 a/1 a/1 s 1 s 2 s 1 s 2 Specification Transfer Fault Mousavi FSM-Based Testing IPM / 64

70 Conformance Testing Solution Homing sequence: ba observed output target state s 2 s 0 s 2 Adaptive distinguishing sequences: DS(s 0 ) = aa, output(s 0, aa) = 00 DS(s 1 ) = aa, output(s 1, aa) = 01 DS(s 2 ) = a, output(s 2, a) = 1 Mousavi FSM-Based Testing IPM / 64

71 Machine Identification Outline 1 Finite State Machines 2 Final State Identification 3 Initial State Identification 4 Conformance Testing 5 Machine Identification 6 Reversibility and Testing Mousavi FSM-Based Testing IPM / 64

72 Machine Identification Basic Idea Implementation: A blackbox B, only input-output observable Problem: by applying a testcase (a sequence of inputs) determine an FSM A such that A = B Same challenges as in conformance checking: B should be strongly connected, finite and constant. But that is not all... Mousavi FSM-Based Testing IPM / 64

73 Machine Identification Simple solution Construct all different reduced and strongly connected FSM s with n states and I B and O B as inputs and outputs, Use conformance testing: find the one conforming to B! Mousavi FSM-Based Testing IPM / 64

74 Machine Identification Towards Automata Learning State-space explosion: with n states, p inputs and q outputs, (nq) np /n! machines 2 states, 2 inputs, 2 outputs, 256 machines! Possible Solutions: Run all possible machines simultaneously (construct direct sum machines) Use machine learning: have an oracle which provides a test-case for each failure Mousavi FSM-Based Testing IPM / 64

75 Machine Identification Towards Automata Learning Mousavi FSM-Based Testing IPM / 64

76 Reversibility and Testing Outline 1 Finite State Machines 2 Final State Identification 3 Initial State Identification 4 Conformance Testing 5 Machine Identification 6 Reversibility and Testing Mousavi FSM-Based Testing IPM / 64

77 Reversibility and Testing State Verification and Reversibility UIO sequence A UIO sequence for a state s is a walk ρ such that ρ produces different output sequence from s than from any other state s. Mousavi FSM-Based Testing IPM / 64

78 Reversibility and Testing State Verification and Reversibility UIO sequence A UIO sequence for a state s is a walk ρ such that ρ produces different output sequence from s than from any other state s. Hardness of State Verification Checking the existence of UIO Sequences is PSPACE-complete. Mousavi FSM-Based Testing IPM / 64

79 Reversibility and Testing State Verification and Reversibility UIO sequence A UIO sequence for a state s is a walk ρ such that ρ produces different output sequence from s than from any other state s. Hardness of State Verification Checking the existence of UIO Sequences is PSPACE-complete. General Idea Once a UIO sequence for a given state is found, one can construct UIOs for other states by taking transitions backwards. Mousavi FSM-Based Testing IPM / 64

80 Reversibility and Testing State Verification and Reversibility General Idea Once a UIO sequence for a given state is found, one can construct UIOs for other states by taking backward transitions. Mousavi FSM-Based Testing IPM / 64

81 Reversibility and Testing State Verification and Reversibility General Idea Once a UIO sequence for a given state is found, one can construct UIOs for other states by taking backward transitions. Mousavi FSM-Based Testing IPM / 64

82 Reversibility and Testing State Verification and Reversibility General Idea Once a UIO sequence for a given state is found, one can construct UIOs for other states by taking backward transitions. Mousavi FSM-Based Testing IPM / 64

83 Reversibility and Testing State Verification and Reversibility General Idea Once a UIO sequence for a given state is found, one can construct UIOs for other states by taking backward transitions. Empirical Evidence In 89% of the cases, UIOs can be constructed using invertible sequences from the UIO of a state. [Hierons and Turker, IEEE TSE 15] Mousavi FSM-Based Testing IPM / 64

84 Reversibility and Testing Chasing Invertible Sequences Mousavi FSM-Based Testing IPM / 64

85 Reversibility and Testing Chasing Invertible Sequences Mousavi FSM-Based Testing IPM / 64

86 Reversibility and Testing Chasing Invertible Sequences Mousavi FSM-Based Testing IPM / 64

87 Reversibility and Testing Invertible sequences and Testing Our order of business Finding a set of states S with UIO sequences; finding maximal (number of) invertible sequences to S. Mousavi FSM-Based Testing IPM / 64

88 Reversibility and Testing Invertible sequences and Testing Our order of business Finding a set of states S with UIO sequences; finding maximal (number of) invertible sequences to S. Proper invertible sequences and UIO sequence An invertible sequence ρ is proper when all its suffixes are invertible sequences. Mousavi FSM-Based Testing IPM / 64

89 Reversibility and Testing Our Past Results in a Nutshell Criteria: Maximizing the Number of Visited States 1 Finding the longest proper invertible sequences: NP-Complete 2 Finding spanning trees of invertible sequences: NP-Complete 3 Finding sets of invertible sequences: PSPace-Complete [Hierons, Kirkedal, MRM, Türker. SOFSEM 17] Mousavi FSM-Based Testing IPM / 64

90 Reversibility and Testing Ongoing Work Finding long invertible sequences Idea: 1 Project the FSM into an properly invertible sub-fsm 2 Reverse the invertible sub-fsms 3 Perform breadth-first search from the state(s) with UIOs 4 For the remaining states (without a proper invertible sequence), perform a subset construction (guided by heuristics) to find invertible sequences Mousavi FSM-Based Testing IPM / 64