micromodels of software declarative modelling and analysis with Alloy lecture 4: a case study MIT Lab for Computer Science Marktoberdorf, August 2002

Transcription

1 micromodels of software declarative modelling and analysis with Alloy lecture 4: a case study Daniel Jackson MIT Lab for Computer Science Marktoberdorf, August 2002

2 on research strategy 2

3 on research strategy I have grown more and more aware that success in science comes not so much to the most gifted, nor the most skillful, nor the most knowledgeable, but rather to the superior strategist and tactician. -- Jack Oliver 2

4 on research strategy I have grown more and more aware that success in science comes not so much to the most gifted, nor the most skillful, nor the most knowledgeable, but rather to the superior strategist and tactician. -- Jack Oliver To make an important discovery, you must study an important problem. -- Peter Medawar 2

5 on research strategy I have grown more and more aware that success in science comes not so much to the most gifted, nor the most skillful, nor the most knowledgeable, but rather to the superior strategist and tactician. -- Jack Oliver To make an important discovery, you must study an important problem. -- Peter Medawar Know your secret weapon. -- Herb Simon 2

6 know where you are knowledge age of discovery time 3

7 4

8 collection of aphorisms at theory.lcs.mit.edu/~dnj/6898/lecture-notes.html Session 20: Hints on Research Strategy 4

9 bakery algorithm 5

10 bakery algorithm why this example? 5

11 bakery algorithm why this example? curiosity -- I hadn t done it before 5

12 bakery algorithm why this example? curiosity -- I hadn t done it before familiarity -- you can compare to Rushby 5

13 bakery algorithm why this example? curiosity -- I hadn t done it before familiarity -- you can compare to Rushby illustrate aspects of Alloy modelling 5

14 bakery algorithm why this example? curiosity -- I hadn t done it before familiarity -- you can compare to Rushby illustrate aspects of Alloy modelling aspects 5

15 bakery algorithm why this example? curiosity -- I hadn t done it before familiarity -- you can compare to Rushby illustrate aspects of Alloy modelling aspects no commitment to fixed topology in model itself 5

16 bakery algorithm why this example? curiosity -- I hadn t done it before familiarity -- you can compare to Rushby illustrate aspects of Alloy modelling aspects no commitment to fixed topology in model itself can easily encode traces in the logic 5

17 bakery algorithm why this example? curiosity -- I hadn t done it before familiarity -- you can compare to Rushby illustrate aspects of Alloy modelling aspects no commitment to fixed topology in model itself can easily encode traces in the logic both invariant reasoning & trace analysis 5

18 bakery algorithm why this example? curiosity -- I hadn t done it before familiarity -- you can compare to Rushby illustrate aspects of Alloy modelling aspects no commitment to fixed topology in model itself can easily encode traces in the logic both invariant reasoning & trace analysis can mitigate effects of finite bounds 5

19 general observations 6

20 general observations Rushby on PVS nothing s easy, but everything s possible 6

21 general observations Rushby on PVS nothing s easy, but everything s possible Jackson on Alloy everything s easy, but nothing s possible 6

22 general observations Rushby on PVS nothing s easy, but everything s possible Jackson on Alloy everything s easy, but nothing s possible not quite it s not always so easy more is possible than you might have guessed 6

23 signatures 7

24 signatures module bakery open std/ord sig Process {} sig Ticket {} sig State { ticket: Process ->? Ticket, part idle, trying, critical: set Process } 7

25 signatures module bakery open std/ord sig Process {} sig Ticket {} sig State { ticket: Process ->? Ticket, part idle, trying, critical: set Process } process in critical phase holds no ticket: hand in ticket when you re being served 7

26 safety condition 8

27 safety condition at most one process is in the critical phase: sig State { ticket: Process ->? Ticket, part idle, trying, critical: set Process } fun Safe (s: State) { sole s.critical } 8

28 transition relation 9

29 transition relation fun Trans (s, s': State, p: Process) { let othertickets = s.ticket[process-p], next = Ord[Ticket].next { p in s.trying othertickets in s.ticket[p].^next p in s'.critical && no s'.ticket[p] } or } 9

30 transition relation fun Trans (s, s': State, p: Process) { let othertickets = s.ticket[process-p], next = Ord[Ticket].next { p in s.trying othertickets in s.ticket[p].^next p in s'.critical && no s'.ticket[p] } or } precondition: p is in trying phase and all other tickets follow its ticket 9

31 transition relation fun Trans (s, s': State, p: Process) { let othertickets = s.ticket[process-p], next = Ord[Ticket].next { p in s.trying othertickets in s.ticket[p].^next p in s'.critical && no s'.ticket[p] } or } precondition: p is in trying phase and all other tickets follow its ticket postcondition: p is in critical phase after, and holds no ticket 9

32 other cases 10

33 other cases fun Trans (s, s': State, p: Process) { let othertickets = s.ticket[process-p], next = Ord[Ticket].next or {p in s.critical p in s'.idle s'.ticket[p] = s.ticket[p]} or {p in s.idle p in s'.trying some s'.ticket[p] & othertickets.^next} } 10

34 frame condition 11

35 frame condition define a condition saying that a process p doesn t change: fun NoChange (s, s': State, p: Process) { s.ticket[p] = s'.ticket[p] p in s.idle => p in s'.idle p in s.trying => p in s'.trying p in s.critical => p in s'.critical } 11

36 initial condition 12

37 initial condition fun Init (s: State) { Safe (s) } 12

38 putting things together 13

39 putting things together fun Interleaving () { Init (Ord[State].first) all s: State - Ord[State].last, s : Ord[State].next[s] some p: Process { Trans (s,s',p) all x: Process - p NoChange(s,s',x) } } 13

40 putting things together fun Interleaving () { Init (Ord[State].first) all s: State - Ord[State].last, s : Ord[State].next[s] some p: Process { Trans (s,s',p) all x: Process - p NoChange(s,s',x) } } use of ordering: instantiation imposes a total order on the set State 13

41 allowing simultaneous actions 14

42 allowing simultaneous actions fun Simultaneity () { Init (Ord[State].first) all s: State - Ord[State].last, s': Ord[State].next[s] all p: Process Trans (s,s',p) or NoChange(s,s',p) } 14

43 checking a conjecture 15

44 checking a conjecture assert InterleavingSafe { Interleaving () => all s: State Safe (s) } check InterleavingSafe for 4 but 2 Process 15

45 counterexamples 16

46 how much assurance? 17

47 how much assurance? analysis within bounded scope: check InterleavingSafe for 4 but 2 Process 17

48 how much assurance? analysis within bounded scope: check InterleavingSafe for 4 but 2 Process 2 processes seems reasonable we ve learned something about a real scenario 17

49 how much assurance? analysis within bounded scope: check InterleavingSafe for 4 but 2 Process 2 processes seems reasonable we ve learned something about a real scenario 4 tickets? 4 states? not at all reasonable running out of tickets is a poor approximation not considering all states may miss bugs 17

50 when is a trace long enough? 18

51 when is a trace long enough? for safety properties, check all traces but how long? ie, what is scope of State? 18

52 when is a trace long enough? for safety properties, check all traces but how long? ie, what is scope of State? idea: bound the diameter if all states reached in path k enough to consider only traces k 18

53 when is a trace long enough? for safety properties, check all traces but how long? ie, what is scope of State? idea: bound the diameter if all states reached in path k enough to consider only traces k strategy ask for loopless trace of length k+1 if none, then k is a bound tighter bounds possible: eg, no shortcuts 18

54 when is a trace long enough? for safety properties, check all traces but how long? ie, what is scope of State? idea: bound the diameter if all states reached in path k enough to consider only traces k strategy ask for loopless trace of length k+1 if none, then k is a bound tighter bounds possible: eg, no shortcuts like bounded model checking but can express conditions directly 18

55 when is a trace long enough? for safety properties, check all traces but how long? ie, what is scope of State? idea: bound the diameter if all states reached in path k enough to consider only traces k strategy ask for loopless trace of length k+1 if none, then k is a bound tighter bounds possible: eg, no shortcuts like bounded model checking but can express conditions directly diameter = 1 max loopless = 1 18

56 when is a trace long enough? for safety properties, check all traces but how long? ie, what is scope of State? idea: bound the diameter if all states reached in path k enough to consider only traces k strategy ask for loopless trace of length k+1 if none, then k is a bound tighter bounds possible: eg, no shortcuts like bounded model checking but can express conditions directly diameter = 1 max loopless = 1 diameter = 1 max loopless = 5 18

57 finding the diameter 19

58 finding the diameter fun NoRepetitionsI () { Interleaving () no disj s, s': State Equiv (s,s') } fun Equiv (s, s': State) { s.ticket = s'.ticket s.idle = s'.idle && s.critical = s'.critical } run NoRepetitionsI for 3 but 2 Process, 8 State 19

59 can we fix the tickets in the same way? 20

60 can we fix the tickets in the same way? what we want to do bound the ticket scope for fast analysis but know that we never run out of tickets 20

61 can we fix the tickets in the same way? what we want to do bound the ticket scope for fast analysis but know that we never run out of tickets one idea find diameter of machine ensure enough tickets for longest trace 20

62 can we fix the tickets in the same way? what we want to do bound the ticket scope for fast analysis but know that we never run out of tickets one idea find diameter of machine ensure enough tickets for longest trace a better idea ticket allocations with same process order are equivalent so find diameter with respect to ticket ordering and show not all tickets are used 20

63 defining the order 21

64 defining the order introduce process ordering as a new field sig StateWithOrder extends State { precedes: Process -> Process }{ all p, p': Process p->p' in precedes iff ticket[p'] in ^(Ord[Ticket].next)[ticket[p]] } fact {State = StateWithOrder} 21

65 defining state equivalence 22

66 defining state equivalence define equivalence modulo ordering fun EquivProcessOrder (s, s': State) { s.precedes = s'.precedes s.idle = s'.idle && s.critical = s'.critical } 22

67 defining state equivalence define equivalence modulo ordering fun EquivProcessOrder (s, s': State) { s.precedes = s'.precedes s.idle = s'.idle && s.critical = s'.critical } define no repetition constraint fun NoRepetitionsUnderOrderI () { Interleaving () no disj s, s': State EquivProcessOrder (s,s') } 22

68 finding the bounds 23

69 finding the bounds find a diameter run NoRepetitionsUnderOrderI for 7 but 3 Process, 13 State 23

70 finding the bounds find a diameter run NoRepetitionsUnderOrderI for 7 but 3 Process, 13 State check that tickets not all used assert EnoughTicketsI { Interleaving () => Ord[Ticket].last!in State.ticket [Process] } check EnoughTicketsI for 7 but 3 Process, 12 State 23

71 finding the bounds find a diameter run NoRepetitionsUnderOrderI for 7 but 3 Process, 13 State check that tickets not all used assert EnoughTicketsI { Interleaving () => Ord[Ticket].last!in State.ticket [Process] } check EnoughTicketsI for 7 but 3 Process, 12 State so now we know for 3 processes, 12 states and 7 tickets is fully general 23

72 getting full coverage 24

73 getting full coverage finally, we check this check InterleavingSafe for 7 but 3 Process, 12 State 24

74 getting full coverage finally, we check this check InterleavingSafe for 7 but 3 Process, 12 State if no counterexample we have a proof for 3 processes 24

75 what we did 25

76 what we did unbounded model of bakery no fixed number of processes or tickets 25

77 what we did unbounded model of bakery no fixed number of processes or tickets analysis in small finite scope may miss counterexamples 25

78 what we did unbounded model of bakery no fixed number of processes or tickets analysis in small finite scope may miss counterexamples established diameter for 3 processes, 12 states and 7 tickets is enough 25

79 what we did unbounded model of bakery no fixed number of processes or tickets analysis in small finite scope may miss counterexamples established diameter for 3 processes, 12 states and 7 tickets is enough full analysis for bounded topology all scenarios for 3 processes 25

80 summary of Alloy 26

81 summary of Alloy a simple language relational first-order logic signatures for structuring: global relations description is set of constraints 26

82 summary of Alloy a simple language relational first-order logic signatures for structuring: global relations description is set of constraints an effective analysis simulation & checking are instance-finding user provides scope, distinct from model tool reduces Alloy to SAT 26

83 summary of Alloy a simple language relational first-order logic signatures for structuring: global relations description is set of constraints an effective analysis simulation & checking are instance-finding user provides scope, distinct from model tool reduces Alloy to SAT applications a variety of case studies used for teaching in ~15 universities 26

84 challenges: better analysis 27

85 challenges: better analysis improving analysis exploiting equalities? eliminating irrelevant constraints? choosing symmetry predicates? 27

86 challenges: better analysis improving analysis exploiting equalities? eliminating irrelevant constraints? choosing symmetry predicates? mitigating effects of scope data independence: scope of 3 enough? decision procedure for subset? 27

87 challenges: better analysis improving analysis exploiting equalities? eliminating irrelevant constraints? choosing symmetry predicates? mitigating effects of scope data independence: scope of 3 enough? decision procedure for subset? analyzing inconsistency what when no instances are found? might have shown false => property tool might show which constraints used 27

88 challenges: applications 28

89 challenges: applications finding bugs in code extract formula from procedure p(s,s0,s1,,s ) check the conjecture pre(s) && p(s,s0,s1,,s ) => post(s,s ) counterexample is trace 28

90 challenges: applications finding bugs in code extract formula from procedure p(s,s0,s1,,s ) check the conjecture pre(s) && p(s,s0,s1,,s ) => post(s,s ) counterexample is trace build veneers on Alloy on API, or as macro language eg, role-based access control eg, semantic web design 28

91 challenges: case studies 29

92 challenges: case studies source code control model CVS at multiple levels is it correct? 29

93 challenges: case studies source code control model CVS at multiple levels is it correct? meta modelling check consistency of UML metamodel check theorems of Unified Theory? 29

94 challenges: case studies source code control model CVS at multiple levels is it correct? meta modelling check consistency of UML metamodel check theorems of Unified Theory? dynamic topology algorithms reverse path forwarding, eg 29

95 thank you! 30