Distributed computing Time synchronization. Mathieu Delalandre University of Tours, Tours city, France

Transcription

1 Dstbuted computng Tme synchonzaton Matheu Delalande Unvesty of Tous, Tous cty, Fance 1

2 Tme synchonzaton 1. Intoducton 2. Clock synchonzaton 2.1. Unvesal Coodnated Tme (UCT) 2.2. Netwok Tme Potocol (NTP) 2.3. Global Postonng System (GPS) 2.4. Bekeley Algothm 2.5. Refeence Boadcast Synchonzaton (RBS) 2

3 Intoducton (1) Causalty: two events ae causally elated f the natue of behavo of the second one mght have been nfluenced n any way by the fst one. Causalty among events s a key concept n pogammng fo a vaety of poblems: complaton, database systems, web bowses and text edtos, secue systems, fault dagnoss and ecovey, etc. Wthn a centalzed mult-pocesso system, the clock s common to all the pocesses o synchonzed, IPC s mmedate due to synchonzaton mechansms. As a consequence, executon s synchonous and causalty can be easly mplemented. e.g. 3 pocesses P1, P2, P3 on 3 pocessos on a same compute consdeng that P1 must etun esults to P2 befoe P3 etuns esults to P2. pocesso 1 P 2 needs esults at 2160 P 2 needs esults at 2161 P 1 etuns esults at pocesso pocesso 3 P 3 etuns esults at 2161 A synchonous executon s an executon n whch: () pocessos ae synchonzed and the clock dft ate between any two pocessos s null o bounded. () message delvey tmes ae such that they occu n one logcal step o ound. () we can have a known uppe bound on the tme taken by a pocess to execute a step. 3

4 Intoducton (2) Causalty: two events ae causally elated f the natue of behavo of the second one mght have been nfluenced n any way by the fst one. Causalty among events s a key concept n pogammng fo a vaety of poblems: complaton, database systems, web bowses and text edtos, secue systems, fault dagnoss and ecovey, etc. Wthn a dstbuted system f the physcal clocks ae not pecsely synchonzed, the causalty elaton between events may not be accuately captued. As a consequence, executon s manly asynchonous. e.g. 3 pocesses P1, P2, P3 on 3 pocessos on dffeent computes consdeng that P1 must etun esults to P2 befoe P3 etuns esults to P2. pocesso P 2 needs esults at pocesso 2 P 2 needs esults at 2161 P 1 etuns esults at pocesso 3 P 3 etuns esults at 2161 An asynchonous executon s an executon n whch: () thee s no pocesso synchony and thee s no bound on the dft ate of pocesso clocks. () message delays ae fnte but unbounded. () thee s no uppe bound on the tme taken by a pocess to execute a step. 4

5 Intoducton (3) Clock synchonzaton s the pocess of ensung that physcally dstbuted pocessos have a common noton of tme. When clock ae synchonzed, t suppots dstbuted synchonous executon. Thee ae fou man ssues: 1. how to fx the efeence tme t? e.g. UCT, mean tme, etc. 2. s the synchonzaton seve-based? s the seve actve o passve? 3. the consdeed netwok fo the synchonzaton mechansm. 4. how to deal wth the pecson n synchonzaton e.g. message delay estmaton, clock skew, etc. Methods Netwok Tme Potocol Global Postonng System (GPS) Refeence Tme UCT Seve Netwok Pecson passve passve Intenet UHF estmaton message delay clock offset estmaton Bekeley Algothm mean tme actve Ethenet none Refeence Boadcast Synchonzaton (RBS) clock offsets actve Weless clock skew 5

6 Tme synchonzaton 1. Intoducton 2. Clock synchonzaton 2.1. Unvesal Coodnated Tme (UCT) 2.2. Netwok Tme Potocol (NTP) 2.3. Global Postonng System (GPS) 2.4. Bekeley Algothm 2.5. Refeence Boadcast Synchonzaton (RBS) 6

7 Methods Netwok Tme Potocol Global Postonng System (GPS) Refeence Tme UCT Seve Netwok Pecson passve passve Intenet UHF estmaton message delay clock offset estmaton Bekeley Algothm mean tme actve Ethenet none Refeence Boadcast Synchonzaton (RBS) clock offsets actve Weless clock skew 7

8 Unvesal Coodnated Tme (UCT) (1) Tanst of the sun occus when the sun clmbs to a maxmum heght n the sky. Hozon Measuement methods estmate the sola day lengths e.g. wate clocks (-270 JC), atomc clocks (half of 20 th ). Tme subdvson: The Egyptans subdvded daytme and nghttme nto twelve hous each snce at least 2000 BC. In 1267, the medeval scentst Roge Bacon stated the tmes as a numbe of hous, mnutes and seconds. Sola day: nteval between two tansts of the sun. peod technology accuacy -270 wate clock Na sola second length of soladay = (.e ) 1335 mechancal 10-1 s 1928 quatz s 1946 atomc clock s Eath 8

9 Unvesal Coodnated Tme (UCT) (2) Length of sola day The length of sola days changes ove the yea (+/- 25 s) The mean sola day s = The mean sola second s = n = 1 n soladay n soladay = n tme In 1940 s t was establshed that the peod of the eath s otaton slow down. Ths esults n two man consequences: (1) the numbe of days pe yea educe (-300 Mllons of yeas today : -35 days). (2) the day become longe all the tme ( : + 2s). (1) (2) 9

10 Unvesal Coodnated Tme (UCT) (3) In 1948 has been ntoduced the Atomc Clock, based on Cesum 133, to compute tansts of the sun tanstons of Cesum 133 equals 1 sola second at yea Intenatonal Atomc Tme (TAI) s the mean of Atomc Clocks snce 1 st Januay 1958 (begnnng of logcal tme). Snce 1948, to adjust the TAI tme to the sola one we ntoduce leap seconds when sola tme TAI tme > 0,8 TAI second: leap second (+1) We have ntoduced aound 37 leap seconds snce Tme offset t wth the TAI efeence (1948) TAI wth leap seconds s the Unvesal Coodnated Tme (UTC) TAI Tme offset t second TAI seconds 1 day 0 +0,003 TAI seconds Dffuson of UCT Rado Mode Eath satellte Pecson +/- 10ms +/- 0.5ms 1 yea 0 +1,09 TAI seconds 10

11 Unvesal Coodnated Tme (UCT) (4) Clock skewng denotes the extend to whch the fequency dffes fom UCT. Tme of a clock: the tme of a clock n a machne p s gven by the functon C p (t) whee C p (t) = t fo all t s a pefect clock. Offset clock: s the dffeence between the tme epoted by a clock and UCT. The offset at tme t of a clock p s δ =C p (t) - t. Fequency: the Fequency s the ate at whch a clock pogesses. ' The fequency at tme t of a clock p s C ( t) = dc ( t dt. p p ) Clock tme C p (t) fast clock δ > 0 δ < 0 tme t (UCT) pefect clock slow clock dc p ( t) dt Fast clock >1 Pefect clock =1 Slow clock <1 11

12 Unvesal Coodnated Tme (UCT) (5) Clock skewng denotes the extend to whch the fequency dffes fom UCT. Skew: the skew of a clock s the dffeence between the fequency of the clock and the pefect clock. The skew at tme t of a clock p s ' ' dc p( t). ( C p( t) t) = C p( t) 1 = 1 dt fast clock Clock tme C p (t) pefect clock slow clock dc p ( t) dt Fast clock >1 skew ' C p ( t) 1 > 0 0 Pefect clock =1 ' ( t) 1 = Slow clock <1 C p ' ( t) 1 < 0 C p tme t (UCT) 12

13 Unvesal Coodnated Tme (UCT) (6) Clock skewng denotes the extend to whch the fequency dffes fom UCT. dc ( ) Maxmum dft ate: f a skew s bounded by ρ, clock dvege at ate (fequency) n the ange 1 ρ p t 1+ ρ, dt ρ s specfed by the manufactue and s known as the maxmum dft ate. dft ate dc p ( t) dt = 1+ ρ Clock tme C p (t) a clock pefect clock dft ate dc p ( t) dt = 1 ρ tme t (UCT) 13

14 Unvesal Coodnated Tme (UCT) (7) Clock skewng denotes the extend to whch the fequency dffes fom UCT. e.g. a compute synchonzes wth an UCT seve to estmate the clock skew, the synchonzaton s done one tme a day and the fst day the clock s adjusted wth UCT then δ = 0. day Cp(t) h:m:s:ms 17:5:06:0 13:10:50:0 10:58:15:0 12:24:58:0 Cp(t) s t (UCT) h:m:s:ms 17:5:06:0 13:10:49:241 10:58:13:355 12:24:55:323 t (UCT) s , , ,323 δ ms δ = ( t) t C p Skew 1,04917E-05 1,09094E-05 1,10442E-05 = C e. g. ' p dcp( t) ( t) 1 = 1 dt 1, = ,

15 Unvesal Coodnated Tme (UCT) (8) Clock skewng denotes the extend to whch the fequency dffes fom UCT. e.g. a Weless Senso Netwok (WSN) wth one oot and fou standad motes wth a base staton. The motes have a synchonzaton peod of 5 s. The base staton collects the mote tme evey second. At 800 s, a tempeatue vaaton of 15 C s povoked n one of the motes, deceasng the pefomance of % [J.M. Castllo-Seclla, Electoncs Lettes, 2013]. synchonzaton equest Mote 1 tmestamps 10 Mean offset δ µs +15 C Root mote Mote 2 Mote 3 staton 4,1 Mote 4 1,9 tme synchonzaton tme ecod 0 Tme s

16 Unvesal Coodnated Tme (UCT) (9) Clock skewng denotes the extend to whch the fequency dffes fom UCT. Maxmum dft gap: f two clocks ae dftng fom UCT n the opposte decton, afte a tme t they wee synchonzed, they may be as much as 2 ρ t consdeng a same maxmum dft ate. To bound the skew to an offset δ, we must synchonze evey Clock tme C p (t) C 2 (t) δ C 1 (t) faste clock slowe clock C t) C ( t) = δ = (1 + ρ) t (1 ρ) t 2 ( 1 δ t = 2ρ δ 2ρ t tme t UCT 16

17 Methods Netwok Tme Potocol Global Postonng System (GPS) Refeence Tme UCT Seve Netwok Pecson passve passve Intenet UHF estmaton message delay clock offset estmaton Bekeley Algothm mean tme actve Ethenet none Refeence Boadcast Synchonzaton (RBS) clock offsets actve Weless clock skew 17

18 Netwok Tme Potocol (NTP) (1) Netwok Tme Potocol (NTP) defnes an achtectue fo a tme sevce and a potocol to dstbute tme nfomaton ove the Intenet. Tme s povded by a netwok of seves located acoss the ntenet, oganzed as a tee. Seves of hghe statums synchonze tme fom the lowest ones. UCT souce UCT Tme accuacy Hgh statum 1 (pmay seve) Tme Seve statum 2 Tme Seve Tme Seve statum 3 Tme Seve Tme Seve Tme Seve statum n Low 18

19 Netwok Tme Potocol (NTP) (2) Netwok Tme Potocol (NTP) defnes an achtectue fo a tme sevce and a potocol to dstbute tme nfomaton ove the Intenet. Tme s povded by a netwok of seves located acoss the ntenet, oganzed as a tee. Seves of hghe statums synchonze tme fom the lowest ones. The NTP suppots pa-wse exchange, a pa of seves exchanges message beang tmestamps nfomaton. The poblem s that when contactng a seve, message delay wll have outdated the epoted tme. tme seve statum n+1 T 1 T 4 quey UCT seve seve statum n T 2 T 3 Actons (1) seve n+1 sends message (2) seve n eceves message (3) seve n sends T 3,T 2 (4) seve n+1 eceves message and adjusts clock Delays δ T3 T 2 δ T4 T 1 δ s the estmaton fo the delay of exchange. We assume that the popagaton delay between the seve s oughly the same. ( T2 T1 ) + ( T4 T3 ) δ = 2 Applyng local compason (T4 aganst T1, T3 aganst T2) makes sense, δ s then efomulated. δ = ( T4 3 T2 T1 ) ( T 2 ) 19

20 Netwok Tme Potocol (NTP) (3) Netwok Tme Potocol (NTP) defnes an achtectue fo a tme sevce and a potocol to dstbute tme nfomaton ove the Intenet. Tme s povded by a netwok of seves located acoss the ntenet, oganzed as a tee. Seves of hghe statums synchonze tme fom the lowest ones. At (4) synchonzaton s done wth: θ s the clock offset between the seves n and n+1 θ = C ( t ) C 1( t ) n n+ C n ( t) T + 1 t) = 3 C n ( = T +δ 4 θ = T + δ 3 T 4 20

21 Netwok Tme Potocol (NTP) (4) Netwok Tme Potocol (NTP) defnes an achtectue fo a tme sevce and a potocol to dstbute tme nfomaton ove the Intenet. Tme s povded by a netwok of seves located acoss the ntenet, oganzed as a tee. Seves of hghe statums synchonze tme fom the lowest ones. At (4) synchonzaton s done wth: θ s the clock offset between the seves n and n+1 delay of tme (1) f θ 0 then C ( t) C 1( t) n n+ C ( t n+1 ) we apply s a slow clock C n+ 1( t) = Cn+ 1( t) +θ 21

22 Netwok Tme Potocol (NTP) (5) Netwok Tme Potocol (NTP) defnes an achtectue fo a tme sevce and a potocol to dstbute tme nfomaton ove the Intenet. Tme s povded by a netwok of seves located acoss the ntenet, oganzed as a tee. Seves of hghe statums synchonze tme fom the lowest ones. At (4) synchonzaton s done wth: θ s the clock offset between the seves n and n+1 (2) f θ 0 then C ( t) C 1( t) n n+ C ( t n+1 ) s a fast clock Compute cannot backwad n tme, off couse. Then we educe the speed of the seve n+1 dung a tme CPU untl θ equals 0. ahead of tme Clock tme C p (t) θ C p (t) θ UCT = CPU + θ CPU F = F o CPU RED NORM = = θ 1 CPU UCT = CPU CPU + θ UCT tme t UCT 22

23 Netwok Tme Potocol (NTP) (6) Netwok Tme Potocol (NTP) defnes an achtectue fo a tme sevce and a potocol to dstbute tme nfomaton ove the Intenet. Tme s povded by a netwok of seves located acoss the ntenet, oganzed as a tee. Seves of hghe statums synchonze tme fom the lowest ones. e.g. The nteupton sevce at the clent level (statum n+1), T4 T1, s 710 ms The nteupton sevce at the seve level (statum n), T3 T2, s 550 ms The popagaton delay δ s then 80 ms δ = ( T4 3 T2 T1 ) ( T 2 ) Consdeng a seve / clent clock gap T3 T4 = 264 ms, the seve / clent clock offset s -184 ms θ = T + δ 3 T 4 The clent has a fast clock wth θ < 0, we want to fx the synchonzaton at a CPU fequency ate > 0,95, CPU s then 3496 ms CPU = θ = 3496 ms 1 UCT We can fx the eal-tme synchonzaton 3680 ms UCT = CPU + θ = = 3680 ms 23

24 Methods Netwok Tme Potocol Global Postonng System (GPS) Refeence Tme UCT Seve Netwok Pecson passve passve Intenet UHF estmaton message delay clock offset estmaton Bekeley Algothm mean tme actve Ethenet none Refeence Boadcast Synchonzaton (RBS) clock offsets actve Weless clock skew 24

25 Global Postonng System (GPS) (1) Global Postonng System (GPS) s a satellte based dstbuted system: - GPS uses 29 satelltes each cculatng n an obt at a heght of appoxmately km. - each satellte has up to fou atomc clocks, egulaly calbated fom Eath. - a satellte contnually boadcasts ts poston, and tme stamps n each message wth ts local tme. - ths boadcastng allows evey eceve on Eath to compute ts own poston usng at least fou satelltes. (2) eceve estmates dstances d 1, d 2, d 3, d 4 fom S1, S2, S3 and S4 Satellte 1 (S1) (1) S1 sends UCT and poston 1 GPS eceve (1) S4 sends UCT and poston 4 Satellte 4 (S4) (1) (2) ae steps (1) S2 sends UCT and poston 2 Satellte 2 (S2) (1) S3 sends UCT and poston 3 Satellte 3 (S3) 25

26 Global Postonng System (GPS) (2) Global Postonng System (GPS) s a satellte based dstbuted system: - GPS uses 29 satelltes each cculatng n an obt at a heght of appoxmately km. - each satellte has up to fou atomc clocks, egulaly calbated fom Eath. - a satellte contnually boadcasts ts poston, and tme stamps n each message wth ts local tme. - ths boadcastng allows evey eceve on Eath to compute ts own poston usng at least fou satelltes. e.g. n two dmensons y-axs, epesents the heght d 1 s a satellte poston a pont at some gven dstances fom the two satelltes, the hghest pont s gnoed (t s located n the space) d 2 x-axs, a staght lne along the Eath s suface at sea level. 26

27 Global Postonng System (GPS) (3) Global Postonng System (GPS) s a satellte based dstbuted system: - GPS uses 29 satelltes each cculatng n an obt at a heght of appoxmately km. - each satellte has up to fou atomc clocks, egulaly calbated fom Eath. - a satellte contnually boadcasts ts poston, and tme stamps n each message wth ts local tme. - ths boadcastng allows evey eceve on Eath to compute ts own poston usng at least fou satelltes. We assume that the tmestamp fom a satellte s completely accuate, when a message s eceved fom a satellte we have: = ( T + ) T = ( T T ) + whee, s the measued delay by the eceve fom satellte, T s tmestamp of satellte, T s synchonzed tme of the GPS eceve, s the devaton of the eceve (offset), 27

28 Global Postonng System (GPS) (4) Global Postonng System (GPS) s a satellte based dstbuted system: - GPS uses 29 satelltes each cculatng n an obt at a heght of appoxmately km. - each satellte has up to fou atomc clocks, egulaly calbated fom Eath. - a satellte contnually boadcasts ts poston, and tme stamps n each message wth ts local tme. - ths boadcastng allows evey eceve on Eath to compute ts own poston usng at least fou satelltes. As sgnals tavel wth the speed of the lght c, we have: = + ε (( T T + ) m d = c = c ) d m d d = c( T ε = c T ) whee, c s the speed of the lght ( m.s -1 ), m d s the measued (m) dstance fom the satellte, d s the eal () dstance fom the satellte, s the dstance eo esultng of the clock devaton, ε 28

29 Global Postonng System (GPS) (5) Global Postonng System (GPS) s a satellte based dstbuted system: - GPS uses 29 satelltes each cculatng n an obt at a heght of appoxmately km. - each satellte has up to fou atomc clocks, egulaly calbated fom Eath. - a satellte contnually boadcasts ts poston, and tme stamps n each message wth ts local tme. - ths boadcastng allows evey eceve on Eath to compute ts own poston usng at least fou satelltes. The eal dstance s smply computed as d m = d d + ε = ε = c 2 2 ( x x ) + ( y y ) + ( z z ) whee, (x, y, z ) denote the coodnates of satellte, (x, y, z ) denote the coodnates of the eceve, 2 Fndng the poston coesponds to solvng the system of lnea equaton to obtan x, y, z and usng fou satelltes (at least). Ths belongs to the numecal analyss feld, usng algothms such as the matx decomposton o teatve methods. Thus, a GPS measuement wll also gve an account of the of the actual tme by appoxmatng. 29

30 Global Postonng System (GPS) (6) Global Postonng System (GPS) s a satellte based dstbuted system: - GPS uses 29 satelltes each cculatng n an obt at a heght of appoxmately km. - each satellte has up to fou atomc clocks, egulaly calbated fom Eath. - a satellte contnually boadcasts ts poston, and tme stamps n each message wth ts local tme. - ths boadcastng allows evey eceve on Eath to compute ts own poston usng at least fou satelltes. Msc ssues, so fa we have assumed that measuement ae pefectly accuate, they ae not: - GPS does not take nto account the leap seconds (devaton snce 1 st Januay 2006). - atomc clocks n the satelltes ae not pefect n synch. - the poston of a satellte s not known pecsely. - the eceve s clock has a fnte accuacy. - the sgnal popagaton speed s not a constant. - etc. 30

31 Methods Netwok Tme Potocol Global Postonng System (GPS) Refeence Tme UCT Seve Netwok Pecson passve passve Intenet UHF estmaton message delay clock offset estmaton Bekeley Algothm mean tme actve Ethenet none Refeence Boadcast Synchonzaton (RBS) clock offsets actve Weless clock skew 31

32 Bekeley Algothm Bekeley Algothm: ths method s sutable fo system n whch no machne has a tme sgnal. Tme seve s actve, poolng evey computes of the netwok peodcally. Based on the answes, the tme seve computes mean dft and tell to machne how to set up the clocks. It woks n thee steps (1) (2) (3). compute 1 compute 2 (2) λ1 (2) λ2 Clent ( 3) ε1 (3)ε s wth (1)(2)(3) ae events T ( =s,1,2) (1) Ts + λ λ ae compute tmes clock dft between a compute and seve (λ = T-Ts) tme to tansmt a message wth (2) λs = 0 Seve mn max mean clock dft (1) Ts + Clent ( 3) ε 2 Clock λ = T- Ts ε = λ T +ε Ts 3: :05 T (1) T1 3: :05 T2 2: :05 λ n = (2) (3) = 1 λ = + 5 n λ The communcaton delay s bounded n the case of a local netwok: mn max max ε < mn Due to communcaton an eo 2 s ntoduced at (1), when the seve sends the clock value. Rest of exchanges (2) (3) ae computed fom some delta paametes, these steps have no mpact on pecson. 32

33 Methods Netwok Tme Potocol Global Postonng System (GPS) Refeence Tme UCT Seve Netwok Pecson passve passve Intenet UHF estmaton message delay clock offset estmaton Bekeley Algothm mean tme actve Ethenet none Refeence Boadcast Synchonzaton (RBS) clock offsets actve Weless clock skew 33

34 Refeence Boadcast Synchonzaton (RBS) (1) Refeence Boadcast Synchonzaton (RBS) s dedcated to senso netwok whee no tme seve s avalable. The communcaton between nodes must be estcted to save enegy. The RBS algothm looks-lke the Bekeley Algothm, t devates of the two-ways potocol (.e. keep eceve and sende nto synch) by lettng only eceves synchonze. (1) N 1 N 2 (2) (1) A sende boadcasts a efeence message m. (1) (3) (2) A eceve p smply ecods the local tme T p,m that t eceved wth m. N 3 (2) (3) Two nodes p,q can exchange each othe s delvey tmes n ode to estmate the elatve offset. They stoe ths offset, thee s no need to adjust the clocks. Offset 1 M M [ p, q] = k= 1 T p T The mean offset s computed wth a geometc mean, othe metcs could be appled., k q, k 34

35 Refeence Boadcast Synchonzaton (RBS) (2) Refeence Boadcast Synchonzaton (RBS) s dedcated to senso netwok whee no tme seve s avalable. The communcaton between nodes must be estcted to save enegy. The RBS algothm looks-lke the Bekeley Algothm, t devates of the two-ways potocol (.e. keep eceve and sende nto synch) by lettng only eceves synchonze. e.g. Events Message stacks N 1 N 2 N 3 N 1 boadcasts N 11,10 N 11,12 N 3 boadcasts N 2 boadcasts N 3 boadcasts N 1, N 2 synchonze N 31,14 N 11,10 N 31,13 N 31,14 N 21,18 N 31,14 N 21,18 N 32,24 N 31,14 N 21,18 N 32,24 N 11, N 31, N 21, N 32 ae messages 10,12,13,14, etc. ae the ecod tmes N 11,10 N 31,13 N 11,10 N 31,13 N 32,22 N 11,10 N 31,13 N 32,22 N 11,12 N 11,12 N 21,21 N 11,12 N 21,21 N 11,12 N 21,21 Offset Offset [ N, N ] 1 M 1 = T M 1 2 [ N, N ] = ( ) = = 1, k= 1 p, k T q, k whee T 11 =14, T 12 =24, T 21 =13, T 22 =

36 Refeence Boadcast Synchonzaton (RBS) (3) Refeence Boadcast Synchonzaton (RBS) s dedcated to senso netwok whee no tme seve s avalable. The communcaton between nodes must be estcted to save enegy. The RBS algothm looks-lke the Bekeley Algothm, t devates of the two-ways potocol (.e. keep eceve and sende nto synch) by lettng only eceves synchonze. Ctcal path: In RBS, only the eceves synchonze. As a consequence, RBS elmnates the sende-sde uncetanly fom the ctcal path, makng ctcal path moe accuate: sende Message pepaaton Netwok access (1) Popagaton delays (2) Delvey delays (1) the popagaton tme n senso netwok s oughly a constant. (2) the delvey tme at the eceve vaes consdeately less than the netwok access tme. eceve A eceve B Usual ctcal path Ctcal path n RBS 36