Analysis of Bounds on Hybrid Vector Clocks

Transcription

1 Analysis of Bounds on Hybrid Vector Clocks Sorrachai Yingchareonthawornchai 1, Sandeep Kulkarni 2, and Murat Demirbas 3 Michigan State University 1,2 University at Buffalo 3 (OPODIS 2015)

2 Motivation A gap in dealing with time in theory and practice of distributed systems Two extremes: Ignore physical time completely Use NTP (and variations) for synchronizing time between processes

3 Causality as a way of Ordering in Asynchronous Systems What is causality and logical/vector clocks

4 Issues with LC and VC There is an implicit assumption that no out-of-bound communication is possible Typically not satisfied in many systems Use of timeouts violate this requirement

5 Example X > 0 local predicate Y > 0 local predicate Globally, (X > 0 and Y > 0) True?

6 Example X > 0 Time out Y > 0 Globally, (X > 0 and Y > 0) True?

7 Definition of ɛ-hb Event e ɛ-hb f if e physically happened before f ɛ, or e hb f Goal: Design hvc such that Event e ɛ-hb f iff hvc.e < hvc.f

8 Algorithm for HVC Maintain ph.j Action for send: hvc.j.j = ph.j hvc.j.k = max(hvc.j.k, ph.j ɛ) Action for receive: hvc.j.j = ph.j hvc.j.k = max(hvc.j.k, hvc.m.k, ph.j ɛ) If hvc.j.k = hvc.j.j ɛ then hvc.j.k can be inferred and hence omitted

9 Example implicit explicit uncertainty hvc matrix at time t

10 Properties of HVC If ɛ= 0: size of hvc = 1 If ɛ = : size of hvc = n Question: what happens for intermediate values

11 Outline Analytical Model Simulations Practical Implication Concluding Remark

12 Definitions n: number of processes ɛ: uncertainty bound provided by time protocol δ: minimum message delay hvc size depends on communication characteristic We conduct average case analysis for Random Unicast Model

13 Random Unicast Model For each clock tick, each process has probability α to send a unicast message When it does, it sends to any process with equal probability Goal: predict Y( t : n, ɛ, δ, α) the average size of hvc at time t given system parameters (n, ɛ, δ, α) Denote Yɛ(t) as short hand for Y( t : n, ɛ, δ, α) Also, define yɛ(t) = Yɛ(t)/n, i.e., the fraction of hvc over n

14 AnalyticalResults

15 Result 1 The relationship between yɛ(t) and y (t) on average We use y(t) instead of y (t) Intuition: growth rate up = growth rate down Implication: We can only study y(t)

16 Approach for Computing y(t) Initially, hvc.j.k, j!= k, is Ph.j = 0 (physical clock of process j) Maintain hvc.j.k explicitly iff hvc.j.k is non-negative y(t) refers to the (expected value of) fraction of hvc entries that are maintained explicitly.

17 Computing y(t) Evaluate the number of processes that are aware of the clock of some process, say j Red = process explicitly maintains clock of j Green = process does not explicitly maintain clock of j Observations Messages sent by a red process causes recipient to be red Messages sent by a green process do not affect the color of recipient Red process always stays red Key technique: measure the growth rate of red processes over time

18 Computing y(t): observation Compute number of processes that change color from green to red at time t Number of red messages at time t depends on Number of red process at time t - δ Since we consider worst case analysis, we assume that messages sent at t - δ are received at time t Number of green processes received red messages depends on number of green processes at time t How number of red and green processes evolve over time?

19 Computing At time t - δ, number of red processes is Y(t - δ) Number of red messages sent at time t - δ is Y(t - δ) Number of green processes at time t is n-y(t) Expected number of red messages sent to green processes is

20 Computing Throw balls in n-y(t) bins Empty bins correspond to processes that stay green Throwing A balls in B bins, expected number of empty bins is Hence, number of processes that turn from green to red at time t is

21 Main result 2: Delay Differential Equation Using two facts to simplify equation above: We obtain:

22 Main Result 2: Special Case For a special case when δ*α << 1, Phase Transition, t =

23 Simulations Source code available at

24 Simulated Environment Each process has only access to local time protocol, and run random unicast model To send a message at time t Set deliver time to t+δ To receive a message, Check if deliver time is t

25 Main Results (review) Let and Theorem 1: Theorem 2:

26 Measurement Experiment for Theorem 1: While running the program, measure the current active size of hvc for all processes at each time t Experiment for Theorem 2: Run program for sufficiently large t, then compute the average hvc size for a fixed value ɛ Plot average hvc size in long run for each ɛ

27 Theorem 1 Simulation Result

28 Theorem 2: Simulation Result

29 Practical Consideration

30 Phase Transition Phase Transition, ɛ* = This unit is clock ticks Convert to seconds, Let f = number messages per second Let d = message delay (seconds) If f*d << 1, phase transition becomes

31 Implication Example: Message rate 1000 messages per second Delay is 1ms n = 100 Phase transition is ~ seconds Implication: TrueTime API in Google s Spanner vs. NTP

32 Conclusion We have the exact model of HVC for random unicast model We describe the vanilla implementation of HVC Possible optimization includes message aggregation and selectively increase delay. TrueTime API benefits with simple implementation of HVC

33 Thank you Question?