Data Structures in. Sean Cribbs. Friday, April 26, 13

Transcription

1 Data Structures in Sean Cribbs

2 Riak is Eventually-Consistent

3 Eventual Consistency 1 2 Replicated Loose coordination 3 Convergence

4 Eventual is Good Fault-tolerant Highly available Low-latency

5 Consistency? 1 A 2 No clear winner! 3 Throw one out? Keep both? B

6 Consistency? 1 A 2 No clear winner! 3 Throw one out? Keep both? B Cassandra

7 Consistency? 1 A 2 No clear winner! 3 Throw one out? Keep both? B Cassandra Riak & Voldemort

8 Conflicts! B! A!

9 Siblings in Riak HTTP/ Multiple Choices X- Riak- Vclock: a85hygdgygdkbviszmk55zkyehnzwblkinio8kgf2tyvhyikf0ciszunmtbzh YVKbIhEUl +VK4spDFTPxhHzFyqhEoVQz7wkSAGLMGuz6FSocFIUijE3pt5HlsgCAA== Vary: Accept, Accept- Encoding Server: MochiWeb/1.1 WebMachine/1.9.0 (participate in the frantic) Date: Fri, 30 Sep :24:35 GMT Content- Type: text/plain Content- Length: 102 Siblings: 16vic4eU9ny46o4KPiDz1f 4v5xOg4bVwUYZdMkqf0d6I 6nr5tDTmhxnwuAFJDd2s6G 6zRSZFUJlHXZ15o9CG0BYl

10 Siblings in Riak HTTP/ Multiple Choices X- Riak- Vclock: a85hygdgygdkbviszmk55zkyehnzwblkinio8kgf2tyvhyikf0ciszunmtbzh YVKbIhEUl +VK4spDFTPxhHzFyqhEoVQz7wkSAGLMGuz6FSocFIUijE3pt5HlsgCAA== Vary: Accept, Accept- Encoding Server: MochiWeb/1.1 WebMachine/1.9.0 (participate in the frantic) Date: Fri, 30 Sep :24:35 GMT Content- Type: text/plain Content- Length: 102 Siblings: 16vic4eU9ny46o4KPiDz1f 4v5xOg4bVwUYZdMkqf0d6I 6nr5tDTmhxnwuAFJDd2s6G 6zRSZFUJlHXZ15o9CG0BYl list of siblings

11 Siblings in Riak HTTP/ Multiple Choices X- Riak- Vclock: a85hygdgygdkbviszmk55zkyehnzwblkinio8kgf2tyvhyikf0ciszunmtbzhyvkbiheul +VK4spDFTPxhHzFyqhEoVQz7wkSAGLMGuz6FSocFIUijE3pt5HlsgCAA== Vary: Accept, Accept- Encoding Server: MochiWeb/1.1 WebMachine/1.9.0 (participate in the frantic) Date: Fri, 30 Sep :24:35 GMT Content- Type: multipart/mixed; boundary=yinlmzyur9feb17okmytgksylvh Content- Length: YinLMzyUR9feB17okMytgKsylvh Content- Type: application/x- www- form- urlencoded Link: </riak/test>; rel="up" Etag: 16vic4eU9ny46o4KPiDz1f Last- Modified: Wed, 10 Mar :01:06 GMT {"bar":"baz"} - - YinLMzyUR9feB17okMytgKsylvh Content- Type: application/json Link: </riak/test>; rel="up" Etag: 4v5xOg4bVwUYZdMkqf0d6I Last- Modified: Wed, 10 Mar :00:04 GMT {"bar":"baz"} - - YinLMzyUR9feB17okMytgKsylvh Content- Type: application/json Link: </riak/test>; rel="up"

12 Siblings in Riak HTTP/ Multiple Choices X- Riak- Vclock: a85hygdgygdkbviszmk55zkyehnzwblkinio8kgf2tyvhyikf0ciszunmtbzhyvkbiheul +VK4spDFTPxhHzFyqhEoVQz7wkSAGLMGuz6FSocFIUijE3pt5HlsgCAA== Vary: Accept, Accept- Encoding Server: MochiWeb/1.1 WebMachine/1.9.0 (participate in the frantic) Date: Fri, 30 Sep :24:35 GMT Content- Type: multipart/mixed; boundary=yinlmzyur9feb17okmytgksylvh Content- Length: YinLMzyUR9feB17okMytgKsylvh Content- Type: application/x- www- form- urlencoded Link: </riak/test>; rel="up" Etag: 16vic4eU9ny46o4KPiDz1f Last- Modified: Wed, 10 Mar :01:06 GMT {"bar":"baz"} - - YinLMzyUR9feB17okMytgKsylvh Content- Type: application/json Link: </riak/test>; rel="up" Etag: 4v5xOg4bVwUYZdMkqf0d6I Last- Modified: Wed, 10 Mar :00:04 GMT all the values {"bar":"baz"} - - YinLMzyUR9feB17okMytgKsylvh Content- Type: application/json Link: </riak/test>; rel="up"

13 Semantic Resolution Your app knows the domain - use business rules to resolve Amazon Dynamo s shopping cart

14 Semantic Resolution BAD Your app knows the domain - use business rules to resolve Amazon Dynamo s shopping cart

15 Semantic Resolution BAD Your app knows the domain - use business rules to resolve Amazon Dynamo s shopping cart Ad hoc approaches have proven brittle and error-prone

16 Goals Meaningful values Automatic resolution Transparent to user

17 WARNING This is a lot of math. Side effects may include dry mouth, itchy rash, and a desire to go back for a PhD.

18 Monotonic Functions Change in strictly a single direction Consecutive values may be equal Monotonic: Linear, Exponential Non-monotonic: Quadratic, Sinusoidal

19 Monotonic Functions Change in strictly a single direction Consecutive values may be equal Monotonic: Linear, Exponential Non-monotonic: Quadratic, Sinusoidal

20 Monotonic Logic Existing facts are never refuted New facts can be added Knowledge only grows

21 Monotonic Logic Existing facts are never refuted New facts can be added Knowledge only grows monotonicity of entailment

22

23 Bounded Join Semi-Lattice S,,

24 Bounded Join Semi-Lattice S,, S is a set

25 Bounded Join Semi-Lattice S,, S is a set S (minimal element)

26 Bounded Join Semi-Lattice S,, is a least-upper bound function x, y S, z S: x y = z

27 Bounded Join Semi-Lattice S,, is a least-upper bound function x, y S, z S: x y = z x, y S: x S y x y = y partial order

28 Bounded Join Semi-Lattice S,, is a least-upper bound function x, y S, z S: x y = z x, y S: x S y x y = y partial order x S: x = x identity

29 Set Lattice S = all finite sets = set-union = {}

30 Set Lattice {a,b,c,d,e} Time {a,b,c,d} {a,b,c} {b,c} {a,b} {b,c,d} {b,c,d,e} {c,d,e} {c,d} {d,e} {a} {b} {c} {d} {e} S = all finite sets = set-union = {}

31 Vector Clock

32 Vector Clock Vector clock is a lattice...

33 Vector Clock Vector clock is a lattice... S = all vectors of (Actor, Count) pairs = All Actors, each with their max Count = [] (empty vector)

34 Vector Clock Vector clock is a lattice... S = all vectors of (Actor, Count) pairs = All Actors, each with their max Count = [] (empty vector)...but the associated Riak value is nonmonotonic,

35 Vector Clock Vector clock is a lattice... S = all vectors of (Actor, Count) pairs = All Actors, each with their max Count = [] (empty vector)...but the associated Riak value is nonmonotonic,...and the vclock is not meaningful to the client.

36

37 CRDT Flavors Convergent (state-based) One replica updates, then forwards entire state, downstream merges Commutative (operation-based) Only mutations (ops) communicated Needs a reliable broadcast channel

38 CRDT Types Registers LWW, MV Counters Positive, P/N Sets Grow only, Two-Phase, Observed-Remove Graphs 2P-2P Lists Growable-array Collaborative editing Treedoc

39 Garbage! Monotonic types are space-inefficient Garbage-collection is non-monotonic Strong consensus may be needed for GC Related: vector-clock pruning, key deletion

40 Garbage! Counters Dead actors Sets, Graphs Tombstones

41 Theory Into Practice

42 Introducing Riak DT

43 Riak DT Riak Core Application State-based CRDTS Independent storage engine HTTP API CRDT behaviour QuickCheck convergence property

44 CRDT Behaviour new/0 empty CRDT value/1 the resolved value update/3 mutate CRDT merge/2 converge two CRDTs equal/2 compare internal value

45 G-Counter Simple version vector (28 LoC) [{ActorId,Count}] Update: increment actor s count Merge: greatest value per Actor Value: sum of Counts

46 G-Counter new() - > []. value(gcnt) - > sum([cnt {_Act, Cnt} <- GCnt]). equal(va,vb) - > lists:sort(va) =:= lists:sort(vb).

47 PN-Counter { } P = [{a,10},{b,2}], N = [{a,1},{c,5}] 2 x G-Counter (10 + 2) - (1 + 5) = 12-6 = 6 P - N = value

48 Riak DT In Action Bitcask storage per vnode Value / Update FSM per request Webmachine resource(s) e.g. GET /counters/key Read-repair

49 Trade-Offs Update: Primary replica only Secondary/Fallbacks may Merge PW=DW=1 by default Read-before-Write in the request path

50 But wait...

51 Riak KV Heavily-tested Storage engines Multi-datacenter Active anti-entropy Secondary Indexes, MapReduce KV Object = MV-Register

52 MERGE ALL THE CODE!!!

53 CRDTs in KV Store CRDTs as values inside KV Objects Requires allow_mult = true Resolve CRDTs separately from opaque values Counters-only at first

54 KV Counters

55 KV Counters GET /buckets/b/counters/c => 5

56 KV Counters GET /buckets/b/counters/c => 5 POST /buckets/b/counters/c 10

57 KV Counters GET /buckets/b/counters/c => 5 POST /buckets/b/counters/c 10 GET /buckets/b/keys/c

58 KV Counters GET /buckets/b/counters/c => 5 POST /buckets/b/counters/c 10 GET /buckets/b/keys/c Stop doing that.

59 Demo!

60 #crdt #riak