Clock Synchronzaton n WS: from Tradtonal Estmaton Theory to Dstrbuted Sgnal Processng Yk-Chung WU The Unversty of Hong Kong Emal: ycwu@eee.hku.hk, Webpage: www.eee.hku.hk/~ycwu
Applcatons requre clock synchronzaton Event detecton Data Fuson Sleep and wake-up cycle for power management TDD transmsson schedule 2
Challenges? Ideal case My tme s 4:28:2pm ode ode In realty delays exst The delay between rado chp nterpret and the CPU respondng The tme for rado chp to transform the message to EM wave The tme for convertng the receved EM wave nto the orgnal message Fnally sgnal to CPU recepton s completed 3
System model ode A clock slope= Ideal clock Clock model: c () t = t q slope= α ode B clock Two-way message exchange s used to establsh the clock relatonshp between two nodes: q Real tme real_tme real_tme = real_tme d w 2 n tn t 4 3 n tn t,, n = real_tme d w,, n = 2 [ c ( tn ) q ] = [ c ( tn) q ] d w, n 3 4 [ c ( tn ) q ] [ c ( tn ) q ] d w, n 4
Parwse Synchronzaton: Gaussan case [] Synchronze node to node (assume node s the reference) q c t c t w q c t c w 2 ( n ) = ( n) d, n 3 4 ( n ) = ( tn ) d, n Approach : MLE Assume rounds of tme-stamp exchange, we have 2 c ( ) - ( t ) c t w, 2 c ( ) ( ) - / t c t w, d = 4 3 c ( / t) - c ( t) q t d = w, 4 3 c( t) - c( t) w, Tθ z 5
Log-lkelhood functon: dtθ ln f ( t, T θ, d) = ln 2 2 2 2 q s lnear and can be easly estmated: ˆ( ) ( H ) H θ d = T T T ( t d ) Put t back to the log-lkelhood functon, d can be obtaned by maxmzng Dfferentatng ths functon w.r.t. d and set t to zero: Fnally, the clock parameters are recovered from ( d) = ( I T ( T T ) T )( t d) H H 2 2 = P H H ˆ Pt t P d = H 2 P ˆ = /[ θˆ ( dˆ )], ˆ q = [ θˆ ( dˆ )] /[ θˆ ( dˆ )] t 2 2 6
Approach 2: Low complexty estmator d appears n both system model equatons, so we can elmnate t by addng two equatons 2 3 4 [ c ( tn ) c ( tn )] = [ c ( tn ) c ( t )] 2 w n, w n, n Puttng the rounds of message nto a vector form: q 4 2 3 c ( ) ( ) -2 ( t ) c( t ) c t c t w, w, / ' ' = q / 4 2 3 t = Tθ z c ( t ) c ( t ) c ( t ) c ( t ) -2 w, w, The MLE for ths equaton s θˆ = ( T T ) T t ' H ' ' H ' Ths estmator s of lower complexty snce there s no need to compute d But snce we are not estmatng the unknowns from the orgnal equatons, there may be some loss n performance 7
Comparson We have shown theoretcally that the relatve loss of the performance bound of the low complexty estmator w.r.t. to CRB s less than % 8
Parwse synchronzaton under exponental delays [2][3] Approach : MLE Revst the message exchange equatons: q q 2 3 4 c ( tn ) c ( tn) d = w, n, c ( tn ) c ( tn ) d = w, n If w,n and w,n are..d. exponental R.V.s, the lkelhood functon s (wth =/, =q / ) f c t c t c t c t d 2 3 4 ({ ( n), ( n ), ( n ), ( n )} n=,,, ) 2 2 3 4 exp [( c ( tn ) c ( tn )) 2 d c ( tn ) c ( tn)] n= 2 3 4 [ c ( tn ) c ( tn) d 0] [ c ( tn ) c ( tn ) d 0] [ d 0] n= n= = 9
Closed-form estmate of can be obtaned by dfferentatng the above equaton w.r.t. and set t to zero ˆ Puttng back nto the lkelhood functon, we can show that the MLE that maxmzes the profle lkelhood functon s * * * 2 3 [,, d ] = arg max [( c ( tn ) c ( tn )) 2 d],, d n = 2 { c ( tn ) c ( tn) d 0} n= ; d 0 subect to { ( 3 ) ( 4 ) 0} c tn c tn d n= ; d 0 Ths s a lnear programmng problem, and can be solved usng exstng solver, but the worst case complexty s at least ( 3 ) We have also proposed a low complexty algorthm for solvng ths problem, and the worst case only takes () 0
Approach 2: Iteratve weghted medan Add the two message exchange equatons: 2 3 4 [ c ( tn ) c ( tn )] [ c ( tn) c ( tn )] 2 = w, n w, n = T w,n - w,n becomes Laplacan R.V. wth locaton parameter 0 and scale parameter / The log-lkelhood functon s rn, An estmate of and can be obtaned by mnmzng the second term = T sn, ln f ({ T, T }, ) ln T T 2 s, n r, n n= = s, n r, n 2 n= mn T T 2, n= s, n r, n
ow, consder two sub-problems When s fxed, the problem s mn 2 0.5( T T ) n= r, n s, n The soluton s the medan value of the sequence r, n s, n 0.5( T T ) n= When s fxed, the problem s mn T ( T / T 2 ) n= r, n s, n r, n Ths s a weghted medan problem for the data set Smple procedure exsts to compute ths Two steps are teratvely updated Snce the obectve functon s convex, t wll converge to the global optmal soluton r, n s, n r, n T,( T / T 2 ) n= 2
Comparson Iteratve weghted medan has a sgnfcant loss w.r.t. optmal soluton The man reason s that teratve weghted medan method adds the two message exchange equatons together before estmaton Ths s n contrast to Gaussan settng where ths operaton does not lead to sgnfcant loss 3
Proposed low-complexty algorthm has the same performance as LP solver Proposed low-complexty algorthm has the lowest complexty 4
etwork-wde synchronzaton How to extend parwse algorthm to work for networkwde synchronzaton? Tree based Clustered based eed overhead to buld and mantan tree or cluster structure Error accumulaton s quck as number of layers ncreases Vulnerable f gateway node des 5
Fully Dstrbuted Algorthms Approach : Coordnate Descent [4] Add the two-way message exchange equatons to frst elmnate d: 2 3 4 [ c ( t ) c ( t ) 2 q ] [ c ( t ) c ( t ) 2 q ] = w w n n n n, n, n Assume each node perform tmes two-way message exchanges wth each of ts drect neghbors LL({, q },) M = 2 n= = ( ) M 2 3 4 [ c ( tn ) c ( tn ) 2 q ] [ c ( tn) c ( tn ) 2 q ] But ths s non-convex w.r.t. unknowns 2 6
Wth transformaton = / and = q / M M 2 3 4 LL({, } = 2,) [ c ( tn ) c ( tn )] 2 [ c ( tn) c ( tn )] 2 n= = ( ) {, } {, } T T rn, sn, Ths s convex w.r.t. and, and we can alternatvely mnmze ths LL Dfferentatng LL w.r.t. and set t to zero (also w.r.t ), we get two coupled teratve equatons ˆ ( m) = n= ( ) [2 ˆ 2 ˆ ][ T T ] ˆ [ T T T T ] ( m) ( m) {, } {, } ( m) {, } {, } {, } {, } s, n r, n r, n s, n s, n r, n n= ( ) [( T ) ( T ) ] {, } 2 {, } 2 s, n r, n ( ) m ( m) ˆ ( m) {, } {, } ˆ 4 ˆ = [ Ts, n Tr, n ] ˆ ( m) {, } {, } [ Tr, n Ts, n ] 4 ( ) n = ( ) 2 7
Approach 2: Belef Propagaton [5] The two-way tme-stamp message exchange equaton (between node and ) can be put nto matrx form 2 3 4 [ c ( tn ) c ( tn ) 2 q ] [ c ( tn) c ( tn ) 2 q] = w, n w, n c ( t ) c ( t ) - 2 ( ) ( ) -2 w q / / 2 3 q 4 = c ( t ) c ( t ) - 2 c ( t ) c ( t ) -2 w A β A β = z 2 3 4 c t c t, / /,,, w w,,, Margnalzed posteror dstrbuton at node : M M g( β ) p( β ) p( A, A β, β ) dβ... dβ dβ... dβ,, M = {, } E Computatonal demandng, needs centralzed processng 8
Express the ont posteror dstrbuton usng factor graph Factor node: local lkelhood functon or pror dstrbuton f = p( A, A β, β ),,, = ( A β A β, I ) 2,,, f = p( β ) Varable node: Margnal dstrbuton at each node can be obtaned by message passng on factor graph: Message from varable node to factor node m () l () l f ( β ) ( ), mf β f B( )\ f =, 9
Message from factor node to varable node: m ( β ) = β ) β () l ( l ) f m f ( f, d,, In each teraton, messages are updated n parallel wth the nformaton from drect neghbourng nodes Each node computes the belef locally: b ( l) ( β ( l) ) m f ( β ) fb( β ) = As the lkelhood functon s Gaussan, the messages nvolved n ths algorthm keep the Gaussan form ( l) ( l) ( l) m f β β v f, C f,,, ( ) ( ) ( l) ( l) ( l) mf β β v f C f ( ) (, ),,, 20
In practce, each real node computes both knds of messages Informaton passes between real nodes s set as message from factor-tovarable, and nherts the propertes Stll Gaussan. Only mean and covarance needed to be exchanged Updated n parallel by local computaton wth receved mean and covarance messages from neghborng nodes The belef computed at node, wth the receved messages from all drect neghbourng nodes, s stll Gaussan: b ( β) ~ β μ, P where () l () l () () ( ) ( ) P = C f, μ l l l l = P C f, v f, () ( ) ( l ) ( l ) ( l ) () 2
ˆ β ) β μ ( l) l) The estmate at node s β = βb ( d = The q and can be recovered from βˆ after convergence ( ( l) It s generally known that f the FG contans cycles, messages can flow many tmes around the graph, leadng to the possblty of dvergence of BP algorthm Two propertes: BP n ths applcaton converges regardless of network topology, even under asynchronous message update The converged soluton can also be proved to be equal to the centralzed ML soluton 22
Comparson Smulaton settng: 25 nodes, d [8,2], q [-5.5,5.5], 2 [-0.955,.055],, 000 topologes, = 0. ntalzaton=[, 0], =0 reference node 23
BP converges much faster than CD as second order nformaton s ncluded n the messages BP algorthm under asynchronous message exchange 24
Dstrbuted trackng wth DKF [4] Clock parameters may stay constant wthn a short perod of tme But t wll change over tme We can ether redo synchronzaton (throwng away prevous estmates), or we can do trackng If the change s slow, trackng s preferred Re-representaton of clock model: After samplng: c t t p B t 0 ( ) = q ( ) t 0 = 0 q () d l 0 ( ) = 0 [ ( ) ] 0 q [ ( ) ] 0 m= 0 c l l m l ( l) () l Ths term s due to phase nose ( t) = p B'( t) 25
Wrtng the accumulated skew and offset n recursve forms: ' ' ( l) = ( l ) p [ B ( l) B ( l )] ( l) = ( l ) [ ( l) ] Clock parameter evoluton model: x () l Measurement equatons based on two-way message exchange u () l ( l) 0 ( l ) u ( l) 0 = ( l) 0 ( l ) 0u ( l) 0 Gather all measurement equatons for A T, T, = 2 ( l) 2 ( l) V {, } {, } {, } r n s n l z, = C ( ) ( ), x ( ) l v l l l 0 b Gaussan wth zero mean and varance 2p () 26
If all nformaton s gathered n a sngle place, the optmal soluton s centralzed KF KF cannot be mplemented n dstrbuted way snce the Kalman gan matrx contans correlaton among nodes Soluton: Impose a block dagonal structure on the Kalman gan matrx: xˆ ( l l ) = A xˆ ( l l ) b k k k k xˆ ˆ lk lk = x lk lk K lk z, l C, lx( ) lk lk ( ) ( ) ( )( ( )) k K( l ) = arg mn Tr P( l l ) k k k K( l ) s.t. K( l ) = U K ( l ) Ω k M T k = 2 k We can solve for K (l k ) n closed form Each round of trackng ncludes tme-stamp exchange and messages exchange for KF update 27
Intalzaton: T - x but t takes a long tme to converge (0 0) = [ 0], P(0 0)= I CD + bootstrap for covarance estmaton (5 rounds of ntal tme-stamp exchange) 28
Start wth 25 nodes (B=5) If a node fals durng trackng, we smply remove t from the equatons If the node later resume workng, we wll use ts prevously stored clock parameter estmates and covarance matrx T - If a new node suddenly on n, we wll use x(0 0) = [ 0], P(0 0)= I 29
Dstrbuted algorthm under exponental delays [6] The parwse LP problem can be easly extended to network-wde settng: M * * 2 3 4 [ x, d ] = arg max ( c ( tn ) c ( tn )) ( c ( tn ) c ( tn)) 2d xd, = ( ) n= n= 2 c ( tn ) c ( tn) d 0 = d 0 3 4 subect to c ( tn ) c ( tn ) d 0 (, ) E, n,..., Ths s a LP problem, but very large n sze Centralzed soluton s computatonal expensve and has large communcaton overhead 30
Challenge of solvng ths problem n a dstrbuted way: constrants are coupled for parameters at dfferent nodes By ntroducng slack varables w and auxlary replca varables z, we can transform the problem to x, d, w, z M M T d = 2 = ( ) arg mn a x ( 2 ) ( d 0) ( w 0) s.t. B x = z, E x = z, {, } {, } {, } {, } 2 d = z, w = z, {, } {, } 2 3 4 z = 0 (, ) E 4 {, } q= q Ths problem can be solved by ADMM (teratvely mnmzng the augmented Lagrangan functon w.r.t. the unknowns, x, d, w, z, and the Lagrange multplers) 3
Propertes: The resultant algorthm only nvolves local computaton at each node and communcatons wth ts drect neghbors Closed-form expressons are avalable for each update step It wll converge to the centralzed ML soluton Contrast to exstng applcatons of ADMM: Drect applcaton of ADMM to the orgnal LP would not result n dstrbuted algorthm Most exstng applcatons that result n dstrbuted algorthms are for Gaussan lkelhood Most exstng works consder a sngle (or a small set of) common parameter 32
Smulaton results on a 25 nodes network K=5 = CD as ntalzaton help to speed up convergence 33
Conclusons We dscussed clock synchronzaton n wreless sensor networks We started wth parwse synchronzaton: Gaussan case and exponental case Then we dscussed network-wde synchronzaton: CD, BP, ADMM We also dscussed trackng usng dstrbuted KF Future works: BP based dstrbuted algorthm for exponental delays? How about arbtrary dstrbuted delays, asymmetrc delays? 34
References [] Me Leng and Yk-Chung Wu, ``On Clock Synchronzaton Algorthms for Wreless Sensor etworks under Unknown Delay," IEEE Trans. on Vehcular Technology, vol. 59, no., pp. 82-90, Jan 200. [2] Me Leng and Yk-Chung Wu, ``Low Complexty Maxmum Lkelhood Estmators for Clock Synchronzaton of Wreless Sensor odes under Exponental Delays," IEEE Trans. on Sgnal Processng, Vol. 59, no. 0, pp. 4860-4870, Oct 20. [3] Me Leng and Yk-Chung Wu, ``On ont synchronzaton of clock offset and skew for Wreless Sensor etworks under exponental delay," Proceedngs of the IEEE ISCAS 200, Pars, France, pp. 46-464, May 200. [4] Bn Luo and Yk-Chung Wu, ``Dstrbuted Clock Parameters Trackng n Wreless Sensor etwork," IEEE Trans. on Wreless Communcatons, Vol. 2, no. 2, pp.6464-6475, Dec 203. [5] Jan Du and Yk-Chung Wu, ``Dstrbuted Clock Skew and Offset Estmaton n Wreless Sensor etworks: Asynchronous Algorthm and Convergence Analyss," IEEE Trans. on Wreless Communcatons, Vol. 2, no., pp. 5908-597, ov. 203. [6] Bn Luo, Le Cheng, and Yk-Chung Wu, ``Fully-dstrbuted Clock Synchronzaton n Wreless Sensor etworks Under Exponental Delays," Sgnal Processng, Vol. 25, pp. 26-273, Aug 206. http://www.scencedrect.com/scence/artcle/p/s0656846000578 Further related readngs: Yk-Chung Wu, Qasm M. Chaudhar and Erchn Serpedn, ``Clock Synchronzaton of Wreless Sensor etworks," IEEE Sgnal Processng Magazne, Vol. 28, no., pp.24-38, Jan. 20. 35
Jun Zheng and Yk-Chung Wu, ``Jont Tme Synchronzaton and Localzaton of an unknown node n Wreless Sensor etworks," IEEE Trans. on Sgnal Processng, Vol. 58, no. 3, pp. 309-320, Mar 200. Me Leng and Yk-Chung Wu, ``Dstrbuted Clock Synchronzaton for Wreless Sensor etworks usng Belef Propagaton," IEEE Trans. on Sgnal Processng, Vol. 59, no., pp. 5404-544, ov 20. 36