Parameterised! Linearisability Andrea Cerone

Transcription

1 ised! Linearisability Andrea Cerone Joint work with Alexey Gotsman and Hongseok Yang ICALP - Copenhagen, July 8th, 2014

2 A Simple Example Converting a sequential data structure into a concurrent one Trivial Solution:

3 A Simple Example Converting a sequential data structure into a concurrent one Trivial Solution: Works for any implementation of push but it s inefficient (we can do much better )

4 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests

5 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1. push(5) t 1 do push (5) t 2 do push (7) t 2. push(7) t 3.

6 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1. push(5) t 1 do push (5) t 2 do push (7) t 2 t 3. push(7). t 2 wins

7 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1 do push (5) t 1. push(5) t 2 do push (7) push(5) push(7). t 2 push(7). t 2 wins t 3

8 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1 do push (5) t 1. push(5) t 2 do push (7) push(5) push(7). t 2 push(7) t 3. pop FC stack push, not implemented in (abstract methods)

9 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1 do push (5) t 1. push(5) t 2 do push (7) push(5) push(7). t 2 push(7) t 3. pop FC stack push, not implemented in (abstract methods) Guaranteed to have correct behaviour for (almost) any implementation of abstract methods

10 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1 do push (5) t 1. push(5) t 2 do push (7) push(5) push(7). t 2 push(7) t 3. It appears to behave as the naive push, not implemented solution in FC stack (abstract methods) pop Guaranteed to have correct behaviour for (almost) any implementation of abstract methods

11 Second Order Libraries Public methods define the operations made available to clients Abstract methods (without implementation) can be declared L : M 0! M call public method public method returns call abstract method abstract method returns call public abstract method public abstract method returns

12 Second Order Libraries Public methods define the operations made available to clients Abstract methods (without implementation) can be declared call public method L : M 0! M public method returns Code for methods declared here is defined call abstract method abstract method returns call public abstract method public abstract method returns

13 Second Order Libraries Public methods define the operations made available to clients Abstract methods (without implementation) can be declared call public method L : M 0! M public method returns Code for methods declared here is defined call public abstract method call abstract method abstract method returns public abstract method returns Methods with no code associated

14 Second Order Libraries Public methods define the operations made available to clients Abstract methods (without implementation) can be declared call public method L : M 0! M public method returns Code for methods declared here is defined call public abstract method call abstract method abstract method returns public abstract method returns Methods with no code associated

15 Second Order Libraries Public methods define the operations made available to clients Abstract methods (without implementation) can be declared call public method L : M 0! M public method returns Code for methods declared here is defined call public abstract method call abstract method abstract method returns public abstract method returns Methods with no code associated

16 Instantiation The implementation of abstract methods can be specified! by another library L 2 : M! M 0 L 1 : M 0! M 00

17 Instantiation The implementation of abstract methods can be specified! by another library L 1 L 2 : M! M 00

18 Observational Refinement Program: let L in C 1 k kc n only calls public methods implemented in L L has no abstract methods Observational Refinement: inclusion of client traces, for every possible client and every possible library parameter L 1 v obs L 2 : 8Cl.8L : ;!M.Obs(let ( L 1 L) in Cl) Obs(let ( L 2 L) in Cl)

19 Observational Refinement Program: let L in C 1 k kc n only calls public methods implemented in L L has no abstract methods Observational Refinement: inclusion of client traces, for every possible client and every possible library parameter L 1 v obs L 2 : 8Cl.8L : ;!M.Obs(let ( L 1 L) in Cl) Obs(let ( L 2 L) in Cl) Quantification over clients and library parameters A proof technique is needed

20 Specifying the behaviour of libraries Types for Libraries L : M 0! M (t,?call m(z)) M \ M 0 (t,!return m(z)) Actions: Observable behaviour of Libraries (Different entitities execute in different memory spaces) (t, call m 00 (z)) (t,!call m 0 (z)) (t,?return m 0 (z)) M 0 (t, return m 00 (z))

21 Specifying the behaviour of libraries Types for Libraries L : M 0! M (t,?call m(z)) M \ M 0 (t,!return m(z)) Actions: Observable behaviour of Libraries (Different entitities execute in different memory spaces) (t, call m 00 (z)) (t,!call m 0 (z)) (t,?return m 0 (z)) M 0 (t, return m 00 (z)) (t,?call m(z)) (t,!return m(z)) Interactions with the client m 2 M \ M 0

22 Specifying the behaviour of libraries Types for Libraries L : M 0! M (t,?call m(z)) M \ M 0 (t,!return m(z)) Actions: Observable behaviour of Libraries (Different entitities execute in different memory spaces) (t, call m 00 (z)) (t,!call m 0 (z)) (t,?return m 0 (z)) M 0 (t, return m 00 (z)) (t,?call m(z)) (t,!return m(z)) Interactions with the client m 2 M \ M 0 Interactions with the library parameter (t,!call m 0 (z)) (t,?return m 0 (z)) m 2 M 0

23 Specifying the behaviour of libraries Types for Libraries L : M 0! M (t,?call m(z)) M \ M 0 (t,!return m(z)) Actions: Observable behaviour of Libraries (Different entitities execute in different memory spaces) (t, call m 00 (z)) (t,!call m 0 (z)) (t,?return m 0 (z)) M 0 (t, return m 00 (z)) (t,?call m(z)) (t,!return m(z)) Interactions with the client m 2 M \ M 0 Interactions with the library parameter (t,!call m 0 (z)) (t,?return m 0 (z)) m 2 M 0 Abstract methods invoked by the client (not observable) m 2 M \ M 0

24 Specifying the behaviour of libraries Types for Libraries L : M 0! M (t,?call m(z)) M \ M 0 (t,!return m(z)) Actions: Observable behaviour of Libraries (Different entitities execute in different memory spaces) (t, call m 00 (z)) (t,!call m 0 (z)) (t,?return m 0 (z)) M 0 (t, return m 00 (z)) Histories: (t,?call m 1 (z)) (t,!call m 0 (z)) (t,?return m 0 (z)) (t 0,!return m 1 (z 0 1)) Implemented Code Running (t 2,?call m 2 (z 2 )) (t 2,!return m 2 (z 0 2)) Execution of Abstract Method

25 Denotational Semantics of Libraries JLK : histories generated by a library (arbitrary behaviour of client and library parameter) (t,?call m(z)) M \ M 0 (t,!return m(z)) (t,!call m 0 (z)) (t,?return m 0 (z)) (t, call m 00 (z)) M 0 (t, return m 00 (z))

26 Denotational Semantics of Libraries JLK : histories generated by a library (arbitrary behaviour of client and library parameter) Takes place! non-deterministically (t,?call m(z)) M \ M 0 (t,!return m(z)) (t,!call m 0 (z)) (t,?return m 0 (z)) Chosen! non-deterministically (t, call m 00 (z)) M 0 (t, return m 00 (z))

27 (General) Linearisability h 1 v h 2 : h 2 preserves thread-local subhistories of h 1 + the following order of pairs of actions: 1 (t 1,!return m 1 (z 1 )) (t 2,?call m 2 (z 2 )) (t 1,!return m 1 (z 1 )) x := z 1 arg t2 = x (t 2,?call m 2 (z 2 ))

28 (General) Linearisability h 1 v h 2 : h 2 preserves thread-local subhistories of h 1 + the following order of pairs of actions: 1 (t 1,!return m 1 (z 1 )) (t 2,?call m 2 (z 2 )) (t 1,!return m 1 (z 1 )) x := z 1 arg t2 = x (t 2,?call m 2 (z 2 )) 2 (t 2,?return m 2 (z 2 )) x := z 1 arg t2 = x (t 2,?return m 2 (z 2 )) (t 1,!call m 1 (z 1 )) (t 1,!call m 1 (z 1 )) 3 (t 2,?call m 2 (z 2 )) x := z 1 arg t1 = x (t 2,?call m 2 (z 2 )) (t 1,!call m 1 (z 1 )) (t 1,!call m 1 (z 1 )) (t 2,?return m 2 (z 2 )) arg t1 = x (t 2,?return m 2 (z 2 )) 4 x := z 1 (t 1,!return m 1 (z 1 )) (t 1,!return m 1 (z 1 ))

29 (General) Linearisability h 1 v h 2 : h 2 preserves thread-local subhistories of h 1 + the following order of pairs of actions: 1 (t 1,!return m 1 (z 1 )) (t 2,?call m 2 (z 2 )) (t 1,!return m 1 (z 1 )) x := z 1 arg t2 = x (t 2,?call m 2 (z 2 )) 2 (t 1,!call m 1 (z 1 )) (t 2,?return m 2 (z 2 )) (t 1,!call m 1 (z 1 )) x := z 1 arg t2 = x (t 2,?return m 2 (z 2 )) 3 4 (t 2,?call m 2 (z 2 )) (t 1,!call m 1 (z 1 )) (t 1,!return m 1 (z 1 )) (t 2,?return m 2 (z 2 )) (t 1,!call m 1 (z 1 )) (t 1,!return m 1 (z 1 )) x := z 1 arg t1 = x Needs to! x := z 1 arg t1 = x Call Abstract Methods (t 2,?call m 2 (z 2 )) (t 2,?return m 2 (z 2 ))

30 Encapsulated Linearisability L : M 0! M M 0 \ M = ; v e h 1 h 2 : h 2 preserves thread-local subhistories of h 1 + the following order of pairs of actions: 1 (t 1,!return m 1 (z 1 )) (t 2,?call m 2 (z 2 )) (t 1,!return m 1 (z 1 )) x := z 1 arg t2 = x (t 2,?call m 2 (z 2 )) 2 (t 2,?return m 2 (z 2 )) x := z 1 arg t2 = x (t 2,?return m 2 (z 2 )) (t 1,!call m 1 (z 1 )) (t 1,!call m 1 (z 1 ))

31 Contextuality of Linearisability Theorem: L L in : M! M 0 1, L 2 : M 0! M 00 L 1 v L 2 =) ( L 1 L in ) v ( L 2 L in ) L L 1 L 2 v

32 Contextuality of Linearisability Theorem: L L in : M! M 0 1, L 2 : M 0! M 00 L 1 v L 2 =) ( L 1 L in ) v ( L 2 L in ) L L 1 L 2 L in v L in

33 Contextuality of Linearisability Theorem: L L in : M! M 0 1, L 2 : M 0! M 00 L 1 v L 2 =) ( L 1 L in ) v ( L 2 L in ) L L 1 L 2 L in v L in =) Corollary: L 1 v L 2 L 1 v obs L 2

34 Contextuality of Encapsulated Linearisability Theorem: for libraries with no public abstract methods! encapsulated linearisability is preserved by! instantiation of library parameters Corollary: for libraries with no public abstract methods! encapsulated linearisability implies! observational refinement!

35 Linearisability does not suffice for Flat Combiners FC : {m i } i2i! {do mi } i2i a single thread handles all concurrent requests FC # : {m i } i2i! {do mi } i2i calls to abstract methods regulated by a global lock Instantiating the library parameter int m i (){return gettid(); } Instantiating the t 1 : do mi (); k t 2 : do mi ();

36 Linearisability does not suffice for Flat Combiners FC : {m i } i2i! {do mi } i2i a single thread handles all concurrent requests FC # : {m i } i2i! {do mi } i2i calls to abstract methods regulated by a global lock Instantiating the library parameter int m i (){return gettid(); } Instantiating the t 1 : do mi (); k t 2 : do mi (); using FC # : t 1 : do mi (); k t 2 : do mi (); always returns t 1 always returns t 2 using FC : t 1 : do mi (); k t 2 : do mi (); can return t 2 can return t 1

37 Linearisability does not suffice for Flat Combiners FC : {m i } i2i! {do mi } i2i a single thread handles all concurrent requests FC # : {m i } i2i! {do mi } i2i calls to abstract methods regulated by a global lock Instantiating the library parameter int m i (){return gettid(); } Instantiating the t 1 : do mi (); k t 2 : do mi (); FC 6v Obs FC # =) FC 6v FC # using FC # : t 1 : do mi (); k t 2 : do mi (); always returns t 1 always returns t 2 using FC : t 1 : do mi (); k t 2 : do mi (); can return t 2 can return t 1

38 Properties Satisfied by Flat Combiners FC # linearises FC if the library parameter does not use thread local information 1 (t 1,!return m 1 (z 1 )) (t 2,?call m 2 (z 2 )) (t 1,!return m 1 (z 1 )) x := z 1 arg t2 = x (t 2,?call m 2 (z 2 )) 2 t 1 (,!call m(z)) ( t 1,?return m(z 0 )) =) t2 ( t 2,?return m(z 0 )) (,!call m(z))

39 Up-to Linearisability and Observational Refinement M 0! M,M 0 \ M = ; R : binary relation between sequences of calls and returns to methods in M 0 h 1 v R h 2 : h 1 ClAct v h 2 ClAct h 1 AbsAct R h 2 AbsAct

40 Up-to Linearisability and Observational Refinement M 0! M,M 0 \ M = ; R : binary relation between sequences of calls and returns to methods in M 0 h 1 v R h 2 : h 1 ClAct v h 2 ClAct h 1 AbsAct R h 2 AbsAct (a variant of contextuality holds for v R ) Soundness: L v R 1 v R L =) 2 L 1 obsl 2 only R- closed library parameters are considered

41 Up-to Linearisability and Observational Refinement M 0! M,M 0 \ M = ; R : binary relation between sequences of calls and returns to methods in M 0 h 1 v R h 2 : h 1 ClAct v h 2 ClAct h 1 AbsAct R h 2 AbsAct (a variant of contextuality holds for v R ) Soundness: L v R 1 v R L =) 2 L 1 obsl 2 only R- closed library parameters are considered FC v Rt FC # =) FC v R t obs FC# R t : equivalence of histories up-to thread identifiers

42 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1 do push (5) t 1. push(5) t 2 do push (7) push(5) push(7). t 2 push(7) t 3. It appears to behave as the naive push, not implemented solution in FC stack (abstract methods) pop Guaranteed to have correct behaviour for (almost) any implementation of abstract methods

43 Flat Combiners (Hendler et al, 2010) Idea: let a single thread handle all requests Stack FC stack provides methods push, pop to clients : access to methods in Stack regulated by, do push do pop 0 1 t 1 do push (5) t 1. push(5) t 2 do push (7) Theorem: push(5) push(7) t 2 t 3. push(7). pop FC v R obs FC # FC stack push, not implemented in (abstract methods) Guaranteed to have correct behaviour for (almost) any implementation of abstract methods

44 Conclusions Future Research: Other applications (Joins, Iterators, ) STM and Transactional Boosting Linearisability for Higher Order Objects Connections with other forms of HO-reasoning (e.g. CaReSL)

45 Conclusions Future Research: Other applications (Joins, Iterators, ) STM and Transactional Boosting Linearisability for Higher Order Objects Connections with other forms of HO-reasoning (e.g. CaReSL)

46 Verification of Flat Combiners Goal: FC v Rt FC #

47 Verification of Flat Combiners Goal: FC v Rt FC # Construct the set of histories JFCK Code analysis =) State Transition System Individuation of properties satisfied by h 2 JFCK

48 Verification of Flat Combiners Goal: FC v Rt FC # Construct the set of histories JFCK Code analysis =) State Transition System Individuation of properties satisfied by h 2 JFCK h 2 JFCK! h 0 Changing threads in abstract calls and returns to match corresponding requests from client

49 Verification of Flat Combiners Goal: FC v Rt FC # Construct the set of histories JFCK Code analysis =) State Transition System Individuation of properties satisfied by h 2 JFCK h 2 JFCK! h 0 Changing threads in abstract calls and returns to match corresponding requests from client h 0 7! h 00 h 0 ClAct v h 00 ClAct h 0 AbsAct = h 00 AbsAct

50 Verification of Flat Combiners Goal: FC v Rt FC # Construct the set of histories JFCK Code analysis =) State Transition System Individuation of properties satisfied by h 2 JFCK h 2 JFCK! h 0 Changing threads in abstract calls and returns to match corresponding requests from client h 0 7! h 00 h 0 ClAct v h 00 ClAct h 0 AbsAct = h 00 AbsAct Show that h 00 2 JFC # K

51 Contextuality of Up-to Linearisability L in : M! M 0 rearranging this part of a history! L in according to R causes this part of the history to be rearranged according to G R relates sequences of actions belonging to M 0 \ M G relates sequences of actions belonging to M R G -closure: defines how closure properties of library parameters have to be changed when a inner library is instantiated

52 Contextuality of Up-to Linearisability Theorem L 1 L 2, : M 0! M 00 v R L 1 L 2 L in : M! M 0 L in is R G -closed =) R L in v R G

53 Contextuality of Up-to Linearisability Theorem L 1 L 2 L 1 L 2, : M 0! M 00 v R L in : M! M 0 L in is R G -closed =) M 0 \ M 00 = ; ^ M \ M 0 = ; =) ( L 1 L in ) v G ( L 2 L in ) R L in R R G G v G G

54 Contextuality of Up-to Linearisability Theorem L 1 L 2 L 1 L 2, : M 0! M 00 v R L in : M! M 0 L in is R G -closed =) M 0 \ M 00 = ; ^ M \ M 0 = ; =) ( L 1 L in ) v G ( L 2 L in ) R L in R R G G v G G

55 Remarks on Observational Refinement Observational Refinement! =! May-Testing General Linearisability = Histories Inclusion Bonus Slide!

56 Remarks on Observational Refinement Observational Refinement! =! May-Testing General Linearisability = Histories Inclusion So What About Must Testing? Bonus Slide! Gotsman et al. 2012: Liveness Preserving Atomicity Abstraction Observational Refinement for Liveness Properties (1st order) Sound Proof Technique via a Variant of Linearisability Main Observables: calls/returns, divergence, deadlocks It remains to be seen whether the results scale to 2nd order