The Complexity of Early-Deciding in Unreliable Synchronous Networks Fabrice Le Fessant 1

Transcription

1 The Complexty of Early-Decdng n Unrelable Synchronous Networks Fabrce Le Fessant 1 fabrce@lefessant.net Mcrosoft Research Lab., Cambrdge, CB3 0FB, UK Aprl 1, 2003 Techncal Report MSR-TR Mcrosoft Research Mcrosoft Corporaton One Mcrosoft Way Redmond, WA Work started at INRIA Rocquencourt, B.P. 105, Le Chesnay Cedex, France

2 1 Introducton In most dstrbuted systems, falures are unlkely to occur durng most short dstrbuted operatons, but wll stll occur from tme to tme: thus, dstrbuted algorthms are often desgned to perform very effcently n the absence of falures, whle degradng performances n the presence of falures to preserve some safety propertes. In the doman of agreement algorthms, these algorthms are called early-decdng algorthms, snce they try to reach a decson as fast as possble. Such algorthms, and ther complexty, have already been studed a lot: n [DRS86], Dolev and al. studed such algorthms for non-unform Consensus n dfferent byzantne models; n [CS00], Charron and al. studed the complexty of the Unform Consensus problem n synchronous systems wth crashes, followed by [KR01] for another proof of the same results; n [DFHR02], Delporte and al. presented another early-decdng algorthm for the Global Data Computaton (whch solves both Unform Consensus and NB-AC problems) problem n asynchronous systems wth Perfect Falure Detectors; fnally, [CL02] presented another algorthm for the k-tag problems (see secton 2) n synchronous systems wth crashes. All these papers gave lower-bounds on early-decson or exhbted algorthms to prove the reachablty of these lower-bounds. Whle for classcal algorthms, lower-bounds are expressed as a functon of t, the maxmal number of falures tolerated by the algorthm, for early-decdng algorthms, the lower-bounds are refned to take nto account f, the actual number of falures n a run. We can summarze the results presented n these papers as follows: Byzantne model: Eventually Byzantne Agreement (EBA, equvalent to Consensus) requres mn(f + 2,t + 1) rounds[drs86], and can be solved wthn these bounds wth n > max(4t,2(t + (t 1) 2 )). Smultaneous Byzantne Agreement (SBA) requres mn(n 1, t + 1) rounds [DRS86]. Synchronous model wth crashes: Unform Consensus requres f + 2 rounds f f < t 1, and f + 1 otherwse[cs00, KR01], and can be solved wthn these bounds[cs00]. k-tag (Unform Consensus and NB-AC) requres f +2 rounds f f < t 1, f +1 otherwse[cl02], and can be solved wthn these bounds[cl02]. Synchronous model wth general-omsson falures: Termnatng Relable Broadcast (TRB) requres f + 2 rounds [Ros96a, Ros96b] wth t unbounded. Asynchronous Model wth Perfect Falure-Detectors: Global Data Computaton (Unform Consensus and NB-AC) can be solved mn(f + 2, t + 1) rounds[dfhr02]. In ths paper, we study the complexty of the k-tag agreement problems [CL02] n the synchronous model, wth ether send-omsson or general-omsson falures. In both models, the lower-bounds are dfferent from the crash falures model, only for f = t 1 where f + 2 1

3 rounds are sometmes necessary. It s mportant to notce that the k-tag problems are unform problems, whch are harder to solve n these models than non-unform ones (see [Ros96a] for a smple soluton to the TRB n the general-omsson model). Our man contrbuton s to show that these lower-bounds for early-decdng can be acheved by two dfferent algorthms: the frst algorthm copes wth send-omsson falures, and solves a problem that can be vewed as a generalzaton of [DFHR02], snce the asynchronous model wth perfect falure detecton can be reduced to a synchronous model where ncorrect processes perform send-omssons only one round before ther crash. The second algorthm copes wth general-omsson falures, and requres a strct majorty of correct processes, whch s anyway the requrement of any unform agreement algorthm n that model [BN92]. The next secton presents the context of ths paper (the models and the k-tag class of agreement problems). Secton 3 and 4 respectvely descrbe the algorthms for the sendomsson and general-omsson models, together wth ther proofs. 2 Context 2.1 Model We consder a synchronous model for dstrbuted systems, as descrbed n [Lyn96] wth ether send-omssons or general-omssons. In these models, Π s a fully connected graph wth n nodes. A synchronous algorthm A s a collecton of n processes, one for each node of Π. Each process p, Π, s defned by a set of states states, a subset start states of ntal states, a message-generaton functon msgs mappng states Π to a unque (possbly null message, and a state transton functon trans mappng states and vectors ndexed by Π of messages to states. An executon of the algorthm A begns wth each process n an arbtrary start state. Then each process p, n lock-step, repeatedly performs the followng three steps: 1. Apply msgs to the current state to determne the messages to be sent and send these messages 2. Receve the messages whch are sent to p durng the prevous step 3. Apply trans to the current state and the messages that are just receved to obtan the new state. The combnaton of these three steps s called a round of A. A run of A models an executon of the entre system: t s defned by the collecton of the ntal states of processes, and the nfnte sequence of rounds of A that are executed. Wthn ths well-known synchronous model, we focus on two models of falures: Send-omsson falures: ncorrect processes may omt to send some (or all) of ther messages, and they may even crash. General-omsson falures: ncorrect processes may omt to send some messages, they may omt to receve some messages, and they may even crash. In both models, correct processes always send all ther messages n each round, and receve all the messages that were sent to them n the round. An ncorrect process whch crashes stop executng steps of the algorthm. Moreover, t can crash n the mddle of a round, maybe succeedng n sendng only a subset of the messages specfed to be sent. 2

4 In any run, f s the number of ncorrect processes, whle t s the number of ncorrect processes tolerated by the agreement algorthm consdered. 2.2 Agreement Problems In ths paper, we wll focus on the k-tag class of agreement problems [CL02], defned by the followng three condtons: Unform Agreement: No two processes (whether correct or faulty) decde dfferently. Termnaton: All correct processes eventually decde. k-valdty: 1. If at least k processes start wth 0, then 0 s the only possble decson value. 2. If all processes start wth 1 and at most k 1 falures occur, then 1 s the only possble decson value. Ths class of problems contans two well-known agreement problems: Unform Consensus(n- Tag) and Non-Blockng Atomc Commtment(1-Tag). Between them, k-tag problems can model agreement problems where a commt should only be performed f some quorum of processes do agree. The nterest of studyng ths class of problems s that our algorthms can be parameterzed by the value k, to solve any partcular problem n ths class. Contrary to the approach based on reducton between problems, there s no need n our approach to show that the results stll hold through the reducton process. 3 The Send-Omsson Model We start by presentng the lower-bounds for early-decdng n ths model: Theorem 3.1 (Lower-Bounds) Let A be a k-tag algorthm whch tolerates t process crashes. For each f, 0 f t, there exsts a run of A wth at most f crashes n whch at least one process decdes not earler then durng round f +2 f f t 1, and t+1 otherwse. Proof: The proof wll be publshed n another paper[cl03]. Ths result only dffers from the lowerbounds of the crash model[cs00] n the case f = t 1. Intutvely, n the crash model, the optmzaton n the case f = t 1 comes from the knowledge that, f a process fals durng the round t, ether ts state s lost, or another process has receved a message from t, and, snce t must be correct, wll be able to propagate ths state durng round t + 1. In the send-omsson model, the ncorrect process at round t may fal by not sendng ts state to hs neghbors, but mght stll decde, leadng to a non-unform decson f the prevous optmzaton were performed. Theorem 3.1 In Fgure 1, we present an algorthm for k-tag, that acheves these lower-bounds. The algorthm s parameterzed by a functon Φ k as n [CL02]. Before further nvestgatons, we need some defntons: Defnton 3.2 (Round quescent) For any round r > 0, for any process p, a round s sad quescent f and only f Faled (r) = Faled (r 1). 3

5 states rounds N, ntally 0 W (V { }) n, ntally (,,v,, ) where v s p s ntal value ready, ether false, true, ntally false halt, a Boolean, ntally false F aled and Rec F aled Π, ntally decson V {unknown}, ntally unknown msgs f halt and / Faled then (cond 1) send (ready,w[],faled) to all processes trans rounds := rounds + 1 Rec Faled := Faled let J be the set of ndces of processes, from whch a message has arrved let (ready j,w j,faled j ) be the message receved from p j, for each j J Faled := j J Faled j (Π\J) J := Π\Faled f halt and / Faled then (cond 1) j J, W[j] := w j j / J, W[j] := f not ready then (cond 2) W[] := Φ k (W) f F aled = Rec F aled then (cond 3) ready := true let R = {j J ready j } f R Ø then W[] := w mn(r) f rounds = mn( F aled + 2, t + 1) then decson := W[] (cond 4) Fgure 1: A k-tag Algorthm for the Send-Omsson Model 4

6 Intutvely, a round s quescent for a process f t does not detect new falures durng ths round. We also need to pont out some nterestng propertes of ths algorthm: Falures propagaton Falures are propagated between processes by messages, so that f a process has detected a falure, any process that receves some nformaton from ths process (ether drectly or ndrectly) durng the followng rounds wll also be aware of the detected falure. No ncorrect detectons Correct processes never fal to send a message. Consequently, snce process falures are detected when a message s not receved (there are no receveomssons), correct processes are never suspected, even by ncorrect processes. Now, we can prove that ths algorthm satsfes all the propertes of the k-tag problems: Theorem 3.3 (Termnaton) The algorthm satsfes Termnaton, and all correct processes decde n at most mn(f + 2,t + 1) rounds. Proof: Correct processes are never suspected by other processes, so that they never appear n the F aled sets. Then, (cond4) guarantes termnaton for them n the specfed bounds, snce F aled f. Theorem 3.3 Theorem 3.4 (Unform Agreement) The algorthm satsfes Unform Agreement. Lemma 3.5 If at round r 1, a process p k receves messages (ready,w,faled ) and (ready j,w j,faled j ), such that ready and ready j are true, and {,j} J (r) k, then w = w j. Proof: [of Lemma 3.5] By nducton on r: The frst round where t can happen s r = 2. Processes p and p j can send a ready message at round 2 f and only f the frst round has been and j quescent, e f they have receved all the messages sent to them n the frst round. Consequently, W [] (1) = W j [j] (1), and w = w j. Now, we can prove that, f the lemma holds for all rounds before r, t also holds for round r. Suppose for convenence that p and p j become ready at rounds r and r j respectvely wth r rj. r < r j. Snce {, j} J (r) k, p j does not suspect p at round r j < r, so p j receved the ready message from p at round r j. Snce the lemma holds at round r j, all the ready messages accepted by p j carry the same value, so that W j [j] (rj) = w. r = r j. The rounds r and r j are and j quescent: Faled (r) = Faled (r 1) and Faled (r) j = Faled (r 1) j. Snce p and p j do not suspect each other (snce {, j} J (r) k ), both of them receved the messages from each other at round r, so that Faled (r 1) Faled (r) j and Faled (r 1) j Faled (r). Consequently, Faled (r) = Faled (rj) j : p and p j receved the same set of messages at round r, and thus, W [] (r) = Φ k (W (r) ) = Φ k (W (r) j ) = W j [j] (r), so that w = w j. Lemma 3.5 Proof: [of theorem 3.4] Let p be the frst process to decde, and p j another process that decdes later. p decdes v at round r, and p j decdes v j at round r j, wth r j. There are dfferent cases to consder: 5

7 1. r = t + 1 = r j (a) r = Faled (r) + 1, so Faled (r) = t,.e. p knows all ncorrect processes, and s correct.. r s quescent: Faled (r 1) = t. So, p s a correct, and ts r message contanng all the ncorrect processes s receved by all processes. Thus, all the processes that decde are correct, and they all know all ncorrect processes. If some of them were ready before round r, ther messages are receved and they carry the same value whch s decded by all processes. Otherwse, all processes receve the same messages, so they become ready wth the same value to decde.. r s not quescent: f one process s not ready at the end of round r 1 and stll decdes at round r, t must become ready at round r, and we fall back n the prevous case. So, suppose that all processes are ready at least at the end of r 1. All processes receve the ready messages sent at round r by all correct processes, together wth ther own ready message. From lemma 3.5, all these message carry the same value. (b) r = Faled (r) + 2. We can suppose that p s a correct process. Indeed, there s at least one correct process, and ether ths process satsfes the prevous case (and we can then use that case), or t satsfes ths case, snce all processes decde at round t + 1. Snce r = Faled (r) +2, p has at least seen two quescent rounds, thus, t was already ready at round r 1 = t. At round t + 1, all processes receve p message, snce p s correct, and thus must decde the same value (from lemma 3.5). 2. r < t + 1, so r = Faled (r) + 2. p has seen at least two quescent rounds before r, so t was ready at least at the end of round r 1. Let p c be a correct process. Faled c (r 1) Faled (r). Thus, Faled c (r 1) Faled (r) r 2 (r 1) 1. Consequently, p c has seen at least one quescent round before round r 1, and thus was ready at least at the end of round r 1. Durng round r, p receves both ts own ready message and the ready message from p c. From Lemma 3.5, r decdes the same value as p c. At round r j, p j receves also a ready message from p c, snce p c s correct. Consequently, p j decdes the same value as p (ether because t becomes ready at round r j, or because t receves ts own ready message and we can use the same Lemma). Theorem 3.4 We now address the valdty ssue. For that, we lmt the set to whch functons Φ k belong. More precsely, for each k {1,,n} we restrct ourselves to functons Φ k whch satsfy the followng (C k ) condton: (1) Φ k (W) = 0 { : W[] {0, }} k; (2) Φ k (W) {W[] : W[] } otherwse and W (,, ). Note that for every k {1,,n}, there exst functons whch satsfy (C k ). Moreover, part (2) n (C k ) s qute natural here snce Φ k s never appled to (,, ) durng the algorthm. We now prove that condton (C k ) for functon Φ k n Fgure 1 yelds an algorthm whch solves the k-tag problem. Theorem 3.6 (k-valdty) For any functon Φ k : (V { }) n V verfyng (C k ), the algorthm satsfes k-valdty. 6

8 Proof: Frst, t can be proved by a smple nducton that r, p, p j, W (r) [j] s ether or the ntal value of p j. Now, suppose that at least k processes start wth the ntal value 0. Let p 1,, p k denote such processes. For any round r 1 and process p, v [1..k], W (r) [p v ] {0, }; thus, Φ k (W (r) ) = 0, and 0 s the only possble decson value. On the other hand, consder a run of the algorthm n whch all the processes start wth 1, and wth at most k 1 falures. At the frst round, every process p receves at least n k + 1 messages wth value 1, and so r 1, Φ k (W (r) ) = 1. Theorem The General-Omsson Model states rounds N, ntally 0 faled N, ntally 0 W (V { }) n, ntally (,,v,, ) where v s p s ntal value halt, a Boolean, ntally false Faled Π, ntally decson V {unknown}, ntally unknown msgs f halt then send (decson,w) to all processes / Faled trans rounds := rounds + 1 let J be the set of ndces of processes, from whch a message has arrved let (decson j,w j ) be the message receved from p j, for each j J J := J\Faled Faled := Π\J f j J, decson j unknown, then decson := W j else f Faled > t then halt := true else j Π, W[j] := max{w k [j] k J } faled := Faled f j J, W j W, then faled := faled + 1 f rounds {faled + 2, Faled + 3,t + 1} then decson := Φ k (W) Fgure 2: An Early-Decdng Algorthm n the General-Omsson Model In ths secton, we study the lower-bounds for agreement n the general-omsson synchronous model: n ths model, an ncorrect process may perform send-omssons (fal to send some messages), receve-omssons (fal to receve messages correctly sent by other processes) or crash. 7

9 4.1 Lower-Bounds Frst of all, n the synchronous model wth general-omssons, we can present a frst bound: k-tag cannot be solved wthout a strct majorty of correct processes: Theorem 4.1 (Majorty) In the synchronous model wth general-omssons, k-tag cannot be solved f 2t n. Ths theorem s a smple case of the general result of Bazz and Neger [BN92], for coordnaton problems n synchronous systems wth general-omsson falures. Theorem 4.2 (Lower-Bounds) In the general-omsson model, for any k-tag algorthm, there s a run where one process needs mn(f +2,t+1) rounds to decde, f f s the number of falures n the run. Proof: Ths general-omsson model contans the send-omsson model. Thus, the bounds for sendomssons apply to the general-omssons. Theorem An Early-Decdng Algorthm Now, we ntroduce an algorthm that can be used to solve the k-tag problem n the generalomsson synchronous model. As n the prevous case, the algorthm s parameterzed by a functon Φ k, that can be nstancated to solve every k-tag problem. Ths algorthm acheves the lower-bounds of mn(f +2,t+1). The basc dea s to force detecton of ncorrect processes by breakng any communcaton between two processes when one of the two processes does not receve a message. A process can only suspect more than t processes f t s ncorrect, and thus, s forced to halt by the algorthm before decdng. Other ncorrect processes are not rejected, snce they receve enough nformaton to meet the agreement property from the majorty of correct processes. Contrary to our frst algorthm (for send-omssons), ths algorthm also uses a noton closed to quescent rounds, but here, t s used to delay the decson f the nformaton s not unformally dstrbuted among non-suspected processes. Theorem 4.3 (Unform Agreement) The algorthm satsfes Unform Agreement. Let p k be a process, we defne K (r) = {j W j [k] (r) halt (r) j } Lemma 4.4 Let p be a process and r > 0 a round. If / K (r ), halt (r ) and K (r ) Ø, then there exsts a lst (j r ) 0 r r 1, such that j r K (r) and all p jr are dstnct processes. Proof: [of Lemma 4.4] r [0..r ], K (r) Ø (a value can dsappear durng a round, and reappear after). Let r [1..r 1] be a round where K (r) K (r 1). Snce K (r+1) Ø, let p j K (r+1). 1. p j K (r) K (r 1) : Durng round r + 1, p and p j receved at least one message from the same process p j, snce they receved messages from a majorty of them durng ths round. Snce p j doesnot suspect p j at round r + 1, t receved ts vector W (r 1) j (contanng v k ) at round r, and forwarded ths value to p. Absurd snce p / K (r). 2. p j / K (r) : p j has receved the value v k durng round r+1 from a process p j K(r) K (r 1). Snce p j doesnot suspect p j at round r + 1, t receved a message from p j at round r that already contaned v k, thus p j K (r), contradcton. 8

10 Consequently, there s no round r [1..r 1] such that K (r) K (r 1). r [1..r 1], let j r be a process such that j r K (r) (non empty) and j r / K (r 1). Let j 0 K (0). Ths lst (j r ) satsfes the Lemma 4.4. Lemma 4.4 Proof: [of Theorem 4.3] Let r be the frst round where a process decdes. Let p be such a process, t decdes a vector W (r). Let p j be another process that decdes at round r j r on a dfferent vector W (rj) 1. r = r j : Let v k be the ntal value of p k such that K (r) and j / K (r). From Lemma 4.4, there s a lst (j r ) 0 r r 1 such that j r K (r) and all p jr are dstnct processes. Snce j / K (r), all these processes belong to Faled (r) j, so that Faled (r) j r. (a) r = t + 1: Faled (r) j t + 1, mpossble snce p j doesnot halt at round r. (b) r faled (r) j + 2: Faled (r) j faled (r) j j. + 2 Faled (r) j + 2, also mpossble. 2. r < r j : Let v k be the ntal value of p k such that K (r) and j / K (rj) (the opposte case can be shown mpossble by the same proof as above. (a) r = Faled (r) + 2: l J (r), W (r 1) l = W (r). At least one of these processes (they are a majorty) s receved by p j durng the round r, so p j already knows all the values n W (r) at the end of r. (b) r = Faled (r) + 3: If / K (r 2), from Lemma 4.4, there s a lst (j r ) 0 r r 3 such that j r K (r) and all p jr are dstnct processes. All these processes belong to Faled (r 2), so that Faled (r 2) r 2. But r = Faled (r) + 3, Faled (r 2) Faled (r) Faled (r) + 1 whch s mpossble. Thus, K (r 2), and at least on process trusted by p j receves v k from p durng round r 1, and forwards t to p j durng round r. Thus, p j cannot gnore v k. Theorem 4.3 Theorem 4.5 (Termnaton) The algorthm satsfes Termnaton. Proof: Clearly, at rount t + 1, all processes have fnally decded. Theorem 4.5 Theorem 4.6 (Optmalty) All correct processes decde n at most mn(f + 2, t + 1) rounds. Proof: If p s a correct process (t does not suspect a majorty of processes), t decdes at round f aled + 2, F aled + 3 or t + 1, or earler f t receves a message wth decde = true. We need to show that, f f < t 1, (1) there s a round r such that r {faled (r) + 2, Faled (r) + 3}. If r = Faled (r) + 3, we need also to show that (2) Faled (r) < f. (1) Suppose that there s no round r such that r = Faled (r) + 3. r mnmal such that r > Faled (r) +3. r 1 Faled (r 1) +3, snce r s mnmal. Snce Faled (r 1) Faled (r), we have Faled (r 1) + 3 Faled (r) + 3 < r Faled (r 1) + 4, e r = Faled (r 1) + 4, and thus r 1 = Faled (r 1) + 3. Absurd. Consequently, (1) s satsfed. 9

11 (2) r = Faled (r) + 3 (and r < r, r / {faled (r) + 2, Faled (r) + 3}). Snce r 1 Faled (r 1) + 3 and r 1 / {faled (r 1) + 2, Faled (r 1) + 3}, r 1 < Faled (r 1) + 3,.e. r < Faled (r 1) + 4. r = Faled (r) + 3 and r < Faled (r 1) + 4, wth Faled (r) Faled (r 1) mply that Faled (r 1) = Faled (r). Snce r 1 faled (r 1) + 2 and r 1 = Faled (r 1) + 2, there must exst p j such that j J (r 1) and W (r 2) j W (r 1), e p k, such that K (r 1) and j / K (r 2). From Lemma 4.4, there s a lst (j r ) 0 r r 3, of dstnct processes such that j r K (r). Snce j / K (r 2), all these processes belong to Faled (r 2) j Faled (r 1). (a) If p j s correct, these r 2 processes are correctly suspected,.e. f r 2 Faled (r) + 1. (b) If p j s not correct: j / Faled (r) and p s correct (t only suspects ncorrect processes), so that Faled (r) {j} s a subset of the set of ncorrect processes. Consequently, f Faled (r) + 1. In both cases, we have f Faled (r) + 1. Thus the decson s reached n less than f + 2 rounds. Theorem 4.6 Theorem 4.7 (k-valdty) For any functon Φ k : (V { }) n V verfyng (C k ), the algorthm satsfes k-valdty. Proof: The proof s exactly the same as for the k Valdty n the send-omsson model for the prevous algorthm (Theorem 3.6). Theorem Concluson We have presented two early-decdng algorthms for the k-tag class of agreement problems n the synchronous model wth message and crash falures. Both algorthms meet the known lower-bounds of mn(f + 2,t + 1) for early-decdng n ther partcular model: the frst algorthm can be used n the send-omsson model, whereas the second one s to be used n the general-omsson model. In ths latter model, agreement can only be reached f a majorty of processes are correct. These algorthms are partcularly appealng, snce they allow programs to reach agreement wth the best speed (n the number of synchronous rounds), whenever there are falures or not. References [BN92] [CL02] [CL03] R. Bazz and G. Neger. The possblty and the complexty of achevng faulttolerant coordnaton. In Prncples of dstrbuted computng (PODC 92), pages ACM Press, B. Charron-Bost and F. Le Fessant. Valdty condtons n agreement problemsand tme complexty. Techncal Report RR-4526, INRIA, Rocquencourt, August B. Charron-Bost and F. Le Fessant. Agreement problems reducton between models. In preparaton, January

12 [CS00] B. Charron-Bost and A. Schper. Unform consensus s harder than consensus. Techncal Report DSC/2000/028, EPFL, May [DFHR02] C. Delporte, H. Fauconner, J.-M. Helary, and M. Raynal. Early stoppng n global data computaton. Techncal Report 1436, IRISA, Jan [DRS86] [KR01] D. Dolev, R. Reschuk, and H. R. Strong. Early stoppng n Byzantne agreement. Techncal Report RJ5406, IBM Research Laboratory, December I. Kedar and S. Rajsbound. A smple proof of the unform consensus lower bound. July [Lyn96] N. A. Lynch. Dstrbuted Algorthms. Morgan Kaufmann, [Ros96a] [Ros96b] M.-C. Rosu. Early-stoppng termnatng relable broadcast protocol for generalomsson falures. In Prncples of dstrbuted computng (PODC 96), page 209. ACM Press, M.-C. Rosu. An optmal early-stoppng trb protocol for general-omsson falures. Techncal report, Cornell Unversty,