Closed Form Bounds for Clock Synchronization Under Simple Uncertainty Assumptions

Closed Form Bounds for Clock Synchronization Under Simple Uncertainty Assumptions Saâd Biaz Λ Jennifer L. Welch y Key words: Distributed Computing Clock Synchronization Optimal Precision Generalized Hypercube Lower Bound. 1 Introduction Many applications in a distributed system rely on processors having synchronized local clocks. Many studies have been dedicated to algorithms and lower bounds for clock synchronization under various network assumptions. We consider the problem of synchronizing the clocks of processors in a failure-free distributed system when the hardware clocks do not drift but there is uncertainty in the message delays. Our goal is to develop closed form expressions for how closely the clocks can be synchronized. Lundelius and Lynch [5] showed that the local clocks of n processors in a fully connected network with the same uncertainty u on each link cannot be synchronized any more closely than u (1 1 ). They provide n a simple algorithm that achieves this bound. Halpern et al. [3] subsequently extended this work to consider arbitrary topologies in which each directed edge may have a different uncertainty. They established a closed form expression for the optimal synchronization in a tree and a triangle. For the tree, the bound is equal to 1diam, wherediam is the diameter of the tree with respect to the uncertainties on the links. For the general case, they show that the optimal synchronization is the solution of an optimization problem using linear programming techniques, but they do not give a closed form expression. In this paper we address open question 7 in [3]:...it would be interesting to obtain precise formulas for the imprecision for a number of graphs that arise in practice. For an arbitrary undirected topology with arbitrary symmetric uncertainties, we prove a lower bound of 1 diam on the closeness of synchronization achievable, where diam is the diameter of the graph when the edges are weighted with the uncertainties. Taken together with the upper bound of radius (the radius of the graph with respect to the uncertainties) previously shown in [3], our result indicates that the tight bound for any (symmetric) topology is known to within a factor of two, since the radius is at most the diameter. Λ School of Science and Engineering, Al Akhawayn University, Ifrane, MOROCCO. saadb@cs.tamu.edu. Phone: (1) 5586-106. Supported by the Fulbright program. On leave from the Ecole Supérieure de Technologie Fès (MOROCCO). y Department of Computer Science, Texas A&M University, College Station, T 77843. welch@cs.tamu.edu. Phone: (979) 845-5076. Fax: (979) 847-8578. Supported in part by NSF CCR-9396098. 1

We then consider the class of topologies described as k-ary m-cubes, both with and without wraparound. The number of nodes, n, equals k m. We assume that every edge has the same uncertainty, u. For the k-ary m-cube without wrap-around, we show that our lower bound 1diam = 1 um(k 1) is tight, by analyzing the synchronization achieved by a simple algorithm. Since chains, (square) meshes, and hypercubes are special cases of this topology, we have the following tight bounds: for an n-processor chain (k = n; m =1), 1 u(n 1); forap n p n mesh (k = p n; m =), u( p n 1); foralog n-dimensional hypercube (k =;m = log n), 1 u log n. For the k-ary m-cube with wrap-around, we show using the same algorithm that our lower bound 1diam = 1 umb k c is tight when k is even and is almost tight when k is odd. We restrict attention to the case k 3; otherwise wrap-around does not add any additional communication possibilities. When k is even, the tight bound is 1 umk. When k is odd, the lower bound is 1 um(k 4 4 1) and the upper bound is 1 um(k 4 1 k ). This result implies tight or almost tight bounds for rings and tori. In particular, we have the following tight bounds: for n-processor ring with n even (k = n; m = 1), 1 un; 4 forp n p n torus with p n even (k = p n; m = ), 1 up n. The analogous bounds when n is odd are not quite tight. In particular, for the ring the lower bound is 1 u(n 4 1) and the upper bound is 1 u(n 4 1 ); for the torus the lower bound is n 1 u(p n 1) and the upper bound is 1 u(p n p 1 n ). Attiya et al. [1] take a different approach to the problem. They consider arbitrary topologies under different delay assumptions. Instead of trying to establish the optimal closeness of synchronization achievable in a given network, they consider the problem of establishing the tightest clock synchronization for any given execution. In other words, given the information collected by any algorithm strategy, they give the optimal synchronization possible. They show how an external observer who has access to all the views of the processors can compute the optimal adjustment to each clock. Patt-Shamir and Rajsbaum [6] extended this work to provide an on-line algorithm optimal for every execution. In Section, we present our model and review relevant prior results. In Section 3, we establish a lower bound for an arbitrary topology. Section 4 is dedicated to establish closed forms for the upper bounds for k-ary m-cube networks, first for networks without wrap-around and then for networks with wrap-around. We conclude in Section 5 with some open problems. Definitions and Prior Results We first describe our formal model and problem definition (following [], which is based on those in [5] and [3]). Then we summarize previous results upon which our results rely, namely the conversion of any communication network to an equivalent clique and the technique of shifting executions..1 Model We consider a set of n processors p 0 through p, connected via point-to-point links. Each link, say the one between p i and p j, reliably delivers messages from p i to p j and vice versa subject to some arbitrary delay

within a known range. (We assume the range is the same in both directions.) The topology and message delay ranges are represented with an undirected graph G =(V;E), whose nodes represent processors and whose edges represent links, together with two mappings L and H from E to nonnegative reals. L(i; j) is the minimum delay on the edge between p i and p j and H(i; j) is the maximum delay on the edge between p i and p j. (Thus H(i; j) must be at least as large as L(i; j).) The difference, H(i; j) L(i; j), isthe uncertainty on the edge. G =(V;E;L;H) is said to be a communication network. The events that can occur at a processor include the arrival of messages from processors that are neighbors in the given topology, as well as internal happenings. Processor p i has a hardware clock value HC i, which is a function from reals (real time) to reals (clock time) of the form HC i (t) =t+c i for some constant c i. This is a hardware clock with no drift; it runs at the same rate as real time. A processor p i is a state machine with a set of initial states and a transition function. The transition function takes as input the current state, the current value of the hardware clock, and the current event and produces a new state and a set of messages to send to p i s neighbors. The hardware clock is not part of the processor s state and cannot be modified by the processor. A history of processor p i is a sequence of alternating states and (event,hardware clock value) pairs for p i, beginning with p i s initial state. We require that each subsequent state follows correctly, according to p i s transition function, from p i s previous state, event and hardware clock value. A timed history of p i is a history of p i together with an assignment of real times to each (event,hardware clock value) pair. The real times must be consistent with the hardware clock values, i.e., the time t assigned to (e; T ) must satisfy HC i (t) =T.Anexecution (of the entire system) is a set of n timed histories, one per processor. For every pair of processors p i and p j, there must be a bijection from messages sent by p i to messages received by p j (i.e., every message sent is received, and only messages sent are received). The delay of a message is the difference between the real time when it was sent and the real time when it was received. Two executions ff 1 and ff are indistinguishable if, for each processor p i, p i has the same (untimed) history in ff 1 as in ff. An execution ff is admissible for communication network G =(V;E;L;H) if every message between any two processors p i and p j has delay within the interval [L(i; j);h(i; j)].. The Clock Synchronization Problem We now precisely define the problem of synchronizing clocks. We assume that each processor p i has a state component adj i which is used to adjust the clock value. We define the processor s adjusted clock AC i (t) to be equal to HC i (t) +adj i (t) (the second term is the value of adj i at real time t). The algorithm has terminated in an execution at a point if no processor ever changes its adj variable thereafter. Achieving "-Synchronized Clocks (in communication network G): In every execution that is admissible for G, there exists a real time t f such that the algorithm has terminated by real time t f, and, for all processors p i and p j, and all t t f, jac i (t) AC j (t)j»". 3

Given a communication network G,letopt(G) be the smallest " such that "-synchronized clocks can be achieved in G. This quantity is the optimal clock synchronization that can be achieved in the network. We are interested in obtaining tight bounds on opt(g) for various G s..3 The Equivalent Clique If two communication networks G and G 0 satisfy opt(g) = opt(g 0 ), then we say that they are equivalent. Consider any communication network G = (V; E;L;H). Let G 0 be the communication network (V;E 0 ;L 0 ;H 0 ),wheree 0 = V V (i.e., G 0 is a clique), and for all i and j, L 0 (i; j) =0and H 0 (i; j) is the length of the shortest path between p i and p j in G with respect to the uncertainties. Section 6 of [3] shows: Theorem 1 The communication networks G and G 0 are equivalent. We prove our lower bound and analyze our algorithm for cliques, instead of more complicated topologies. Theorem 1 shows that the results obtained on the cliques apply to the original topologies. Note that the uncertainties in G 0 satisfy the triangle inequality, since H 0 is defined using shortest path lengths..4 Shifting Like those in [5] and [3], our lower bound result is obtained by shifting executions, that is, by shifting the real times at which events occur. More formally, let ff =( 0 ; 1 ;:::; )be an execution and let x i be a real number, 0» i» n 1. Defineshift(ff; hx 0 ;:::;x i)to be ( 0 0 ; 0 1 ;:::; 0 ), where each 0 i is the timed history resulting from i by adding x i to the real time associated with each event in i. Shifting an execution changes the hardware clocks and the delays experienced by messages. Lundelius and Lynch [5] quantified the changes as follows: Lemma Let ff be an execution with hardware clocks HC i and let x i be a real number, 0» i» n 1. Then shift(ff; hx 0 ;:::;x i)is an execution that is indistinguishable from ff, and in which (a) the hardware clock of p i,hc 0 i (t), is equal to HC i(t) x i for all t, and (b) every message from p i to p j has delay ffi x i + x j,whereffi is the delay of the message in ff. Note that shifting an execution might not result in an admissible execution. 3 Lower Bound for Arbitrary Topology Let diam(g) denote the diameter of communication network G with respect to the uncertainties. Theorem 3 For any communication network G, opt(g) 1 diam(g). 4

Proof Let G 0 be the clique from Section.3 that is equivalent to G. Letu ij denote H 0 (i; j), the uncertainty on edge (p i ;p j ) in G 0. We will show that opt(g 0 ) is at least 1 diam(g0 ). Since Theorem 1 implies that opt(g 0 )=opt(g) and since the diameter of G is the same as that of G 0, the result follows. Let p a and p b be two processors in G 0 such that u ab equals the diameter of G 0. Choose any algorithm for G 0 that achieves "-synchronized clocks. Consider the admissible execution ff in which the delays are as follows. For any two processors p i and p j,whereu ai < u aj (i.e., p i is at least as close to p a, w.r.t. uncertainties, as p j is), the delay of every message from p i to p j is 0, and the delay of every message from p j to p i is u ij.whenu ai = u aj, we choose the delay of every message from p i to p j to be 0, and the delay of every message from p j to p i to be u ij. Since the algorithm must work correctly in ff, AC a AC b ": (1) Let ff 0 = shift(ff; ~x), wherex i = u ai u ab, 0» i» n 1. Note that p a is shifted by u ab and p b is shifted by 0. By Lemma, the delay of every message in ff 0 from p i to p j is 0 (u ai u ab )+(u aj u ab )=u aj u ai. This expression is at most u ij, by the triangle inequality, so the new delay is not too big. This expression is at least 0, since u ai» u aj, so the new delay is not too small. By Lemma, the delay of every message in ff 0 from p j to p i is u ij (u aj u ab )+(u ai u ab ) = u ij + u ai u aj. This expression is at most u ij,sinceu ai» u aj, so the new delay is not too big. This expression is at least 0, by the triangle inequality, so the new delay is not too small. Therefore ff 0 is admissible. Since the algorithm must work correctly in ff 0,wehave: ACb 0 AC0 a ": () Since AC 0 b = AC b +0and AC 0 a = AC a + u ab, inequalities (1) and () imply that " 1 u ab. Thus for the arbitrary algorithm, " is at least half the diameter. 4 Upper Bounds 4.1 The Algorithm We consider the averaging algorithm of [5] for a clique, slightly modified to handle the fact that different edges have different uncertainties. (The description of the algorithm and Lemma 4 are modeled after [].) Each processor p i begins by sending its current hardware clock value to all the other processors. It has an array diff whose j-th entry is set equal to p i s estimated difference between p i s hardware clock and p j s hardware clock. For j 6= i, this difference is estimated using the value T contained in the message that p i receives from p j and assuming that this message took u ij time to arrive. Once p i has an estimate for 5

initially diff i [i] =0 at first computation step: 1: send HC i (current hardware clock value) to all other processors upon receiving message T from some p j : : diff i [j] :=T+ u ij HC i 3: if a message has been received from every other processor then 4: adj i P := 1 n diff k=0 i[k] Figure 1: Averaging algorithm; code for processor p i, 0» i» n 1. every processor, it sets its adjustment to the average of all the estimated differences. (See Figure 1.) We will analyze the clock synchronization achieved by the averaging algorithm for a clique G 0 with arbitrary symmetric uncertainties (denoted u ij ). Then we will show how the general result can be specialized into closed forms for specific cliques that are equivalent to certain topologies. Theorem 1 shows that the upper bounds obtained for these cliques also hold for the other topologies. The applicability of the averaging algorithm to general topologies requires all-pairs shortest-paths computation. Lemma 4 Consider any execution of the averaging algorithm that is admissible for G 0. For any two processors p i and p j and any time t after the algorithm terminates (executes Line 4), jac i (t) AC j (t)j» 1 n 0 @ l=0;l6=i u li + l=0;l6=j u lj 1 A : Proof For every time t after p i sets diff i [j], diff i [j](t) =HC j (t) HC i (t)+ err ji,whereerr ji is a constant. The definition of the algorithm shows that jac i (t) AC j (t)j» 1 n 0 @ jerrij j + jerr ji j + The bounds on err ji and the rules of absolute value give the result. l=0;l6=i;j 1 jerr li err lj ja : 4. Generalized Hypercubes Without Wrap-around The k-ary m-cube network is a generalization of the hypercube architecture where k =. Inthek-ary m-cube, there are m dimensions and k nodes on each dimension (see [4], p. 86). More formally, each node is represented as a vector ha 0 ;a 1 ;:::;a m 1 i, where each a r is an element of f0; 1;:::;k 1g. Thus there 6

are n = k m nodes. There is an edge between two nodes ~a and ~ b if they differ in exactly one component, say the r-th, and a r = b r +1. We need the following fact about distances in (unweighted) k-ary m-cubes without wrap-around. Given such a graph G, letdist G (~a; ~ b) be the minimum number of edges in any path from ~a to ~ b in G. Lemma 5 For any k-ary m-cube G =(V; E) without wrap-around, for any node ~a in V, dist G (~a; ~ 1 b)» m(km+1 k m ): ~ b Proof Consider a particular node ~ b. The distance between ~a and ~ b is the sum, over all dimensions P r, ofthe distance from a r to b r. With no wrap-around, this distance is jb r a r j. It is obvious that ~ b dist G (~a; ~ b)» P~ b dist G (~ 0; ~ b) where ~ 0 is the vector of all zeroes. Let us then compute P ~ b dist G (~ 0; ~ b). This sum can be written: m 1 ~ b r=0 b r : (3) Now let us concentrate on dimension r. For how many vectors ~ b is b r equal to 0? There are k independent choices for each of the m 1 other entries, so the answer is k m 1. The same is true for every value of b r between 0 and k 1. Thus the contribution to the sum in expression (3) from dimension r is k m 1 k 1 h=0 This sum can be written as 1 (km+1 k m ). We multiply this expression by m, to account for all the dimensions. Theorem 6 For any k-ary m-cube (without wrap-around) communication network G with uncertainty u on each edge, opt(g) is at most 1 um(k 1). Proof Let G 0 be the equivalent clique communication network from Section.3. By Theorem 1, it suffices to show that the averaging function on G 0 achieves "-synchronized clocks, where " = 1 um(k 1). Consider any execution of the algorithm that is admissible for G 0. By Lemma 4, for any two processors p i and p j and any time t after the algorithm terminates jac i (t) AC j (t)j» 1 n h: 0 @ l=0;l6=i u li + l=0;l6=j u lj 1 A : By the definition of G 0, u lh = u dist G (l; h), forh = i; j. By Lemma 5, for h = i; j, Since n = k m, the result follows. l=0;l6=h u lh» um 1 (km+1 k m ): 7

4.3 Generalized Hypercubes With Wrap-around The definition of the k-ary m-cube with wrap-around is the same as for the k-ary m-cube without wraparound, except that we add mod k to the condition under which there is an edge between two nodes. That is, there is an edge between two nodes ~a and ~ b if they differ in exactly one component, say the r-th, and if a r = b r + 1 mod k. Given a graph G, letdist G (~a; ~ b) be the minimum number of edges in any path from ~a to ~ b in G. We need the following fact about distances in (unweighted) k-ary m-cubes with wrap-around. Lemma 7 For any k-ary m-cube G =(V; E) with wrap-around, for any node ~a in V, ( 1 dist G (~a; ~ 4 b)= mkm+1 when k is even 1 ~ b 4 m(km+1 k m 1 ) when k is odd: Proof Consider a particular node ~ b. The distance between ~a and ~ b is the sum, over all dimensions r, ofthe distance from a r to b r. However, we must take into account the wrap-around edges. In this case the distance is jb r a r j if this quantity is at most b k c, otherwise it is k jb r a r j. Thus the desired sum can be written: m 1 ~ b r=0 min(jb r a r j;k jb r a r j): (4) Now let us concentrate on dimension r. For how many vectors ~ b is b r equal to 0? There are k independent choices for each of the m 1 other entries, so the answer is k m 1. The same is true for every value of b r between 0 and k 1. Thus the contribution to the sum in expression (4) from dimension r is k m 1 k 1 h=0 min(jh a r j;k jh a r j): (5) This sum contains two terms equal to 1 (there are two nodes at distance 1 from ~a along dimension r), two terms equal to, etc., up to two terms equal to b k 1 c, and, if k is even, one term equal to k (there is one node at distance k from ~a along dimension r). First consider the case when k is odd. Then expression (5) becomes k m 1 b k 1 c j=1 Algebraic manipulations reduce this to 1 4 (km+1 k m 1 ). We multiply this expression by m, to account for all the dimensions. Now consider the case when k is even. Expression (5) now becomes k m 1 k 1 j=1 j: j + k m 1 k : Algebraic manipulations reduce this to 1 4 km+1. We multiply by m to account for all dimensions. 8

Theorem 8 For any k-ary m-cube (with wrap-around) communication network G with uncertainty u on each edge, opt(g) is at most 1 umk when k is even and 1 um(k 4 4 1 k ) when k is odd. Proof The proof is essentially the same as that for Theorem 6. The differences are that " = 1 umk when k 4 is even and 1 um(k 4 1 k ) otherwise, and that Lemma 7 is invoked instead of Lemma 5. 5 Open Questions The obvious open question in our work is to tighten the bounds for k-ary m-cubes with wrap-around when k is odd. We believe that the algorithm is optimal and that a more involved shifting argument is needed to strengthen the lower bound. The evidence for this is that a 3-ary 1-cube is a ring with three processors, which is also a clique. Lundelius and Lynch [5] showed in this case that the tight bound is u, which is what 3 the algorithm gives, and not 1 u as given by the lower bound of this paper. It would be interesting to characterize the class of networks and uncertainty assumptions for which the simple algorithm is optimal, and similarly those for which the 1 diam lower bound is tight. Further work is needed to obtain closed form expressions for other networks and delay assumptions. References [1] H. Attiya, A. Herzberg, and S. Rajsbaum, Optimal Clock Synchronization Under Different Delay Assumptions, Proc. of the 1th Annual ACM Symp. on Principles of Distributed Computing, pp. 109-10, 1993. [] H. Attiya and J. L. Welch, Distributed Computing: Fundamentals, Simulations and Advanced Topics, (Section 6.3), McGraw-Hill Publishing Company, 1998. [3] J. Y. Halpern, N. Megiddo, and A. A. Munshi, Optimal Precision in The Presence of Uncertainty, Journal of Complexity, vol. 1, pp. 170-196, 1985. [4] K. Hwang, Advanced Computer Architecture, McGraw-Hill Inc., New York, 1993. [5] J. Lundelius and N. A. Lynch, An Upper and Lower Bound for Clock Synchronization, Information and Control, vol. 6, pp. 190-04, 1984. [6] B. Patt-Shamir and S. Rajsbaum, A Theory of Clock Synchronization, Proceedings of the 6th Annual ACM Symposium on Theory of Computing, pp. 810-819, 1994. 9