ROBUST CLOCK SYNCHRONIZATION IN WIRELESS SENSOR NETWORKS. A Thesis SAWIN SAIBUA

ROBUST CLOCK SYNCHRONIZATION IN WIRELESS SENSOR NETWORKS A Thesis by SAWIN SAIBUA Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE August 00 Major Subject: Electrical Engineering

Robust Cloc Synchronization in Wireless Sensor Networs Copyright 00 Sawin Saibua

ROBUST CLOCK SYNCHRONIZATION IN WIRELESS SENSOR NETWORKS A Thesis by SAWIN SAIBUA Submitted to the Office of Graduate Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Approved by: Chair of Committee, Committee Members, Head of Department, Erchin Serpedin Deepa Kundur Aydin I. Karsilayan Radu Stoleru Costas Georghiades August 00 Major Subject: Electrical Engineering

iii ABSTRACT Robust Cloc Synchronization in Wireless Sensor Networs. (August 00) Sawin Saibua, B.Eng., Chulalongorn University Chair of Advisory Committee: Dr. Erchin Serpedin Cloc synchronization between any two nodes in a Wireless Sensor Networ (WSNs) is generally accomplished through exchanging messages and adjusting cloc offset and sew parameters of each node s cloc. To cope with unnown networ message delays, the cloc offset and sew estimation schemes have to be reliable and robust in order to attain long-term synchronization and save energy. A joint cloc offset and sew estimation scheme is studied and developed based on the Gaussian Mixture Kalman Particle Filter (GMKPF). The proposed estimation scheme is shown to be a more flexible alternative than the Gaussian Maximum Lielihood Estimator (GMLE) and the Exponential Maximum Lielihood Estimator (EMLE), and to be a robust estimation scheme in the presence of non-gaussian/nonexponential random delays. This study also includes a sub optimal method called Maximum Lielihood-lie Estimator (MLLE) for Gaussian and exponential delays. The computer simulations illustrate that the scheme based on GMKPF yields better results in terms of Mean Square Error (MSE) relative to GMLE, EMLE, GMLLE, and EMLLE, when the networ delays are modeled as non-gaussian/non-exponential distributions or as a mixture of several distributions.

iv ACKNOWLEDGEMENTS I would lie to than my committee chair, Dr. Serpedin, and my committee members, Dr. Kundur, Dr. Karsilayan, and Dr. Stoleru, for their guidance and support throughout the course of this research. I also would lie to extend my gratitude to Dr. Jangsub Kim and Jaehan Lee who sacrificed their time to help me on my research. Thans also go to my friends and colleagues and the department faculty and staff for maing my time at Texas A&M University a great experience. Finally, thans to my parents for their great encouragement and love.

v TABLE OF CONTENTS Page ABSTRACT... ACKNOWLEDGEMENTS... TABLE OF CONTENTS... LIST OF FIGURES... iii iv v vii CHAPTER I INTRODUCTION.... Wireless Sensor Networs and Their Applications. Importance of Robust Cloc Synchronization.3 Literature Review.. 4.4 Problem Description.. 6.5 Research Objective.... 7 II SYSTEM DESCRIPTION AND BACKGROUND THEORIES... 8. Two-way Message Exchange Mechanism Without Cloc Sew... 8. Maximum Lielihood Cloc Offset Estimation..... 9.. Gaussian Delay Model. 9.. Exponential Delay Model. 0.3 Two-way Message Exchange Mechanism with Cloc Offset and Sew.. 0.4 Joint Maximum Lielihood Estimation (JMLE) of Cloc Offset and Sew....4. Gaussian Delay Model..4. Exponential Delay Model..5 Maximum Lielihood-Lie Estimator (MLLE) 3.5. Gaussian Delay Model. 4.5. Exponential Delay Model. 5

vi CHAPTER Page.6 Cloc Offset Estimation Based on the Gaussian Mixture Kalman Particle Filter...... 7.6. Problem Modeling.... 7.6. Framewor of GMKPF.... 9.6.3 GMKPF Algorithm 3.7 Joint Cloc Offset and Sew Estimation Based on GMKPF. 5.7. Problem Modeling 5 III PERFORMANCE OF GMKPF 8 3. Cloc Offset Estimation Based on GMKPF. 8 3. Combination of Cloc Offset and Sew Estimation Based on GMKPF. 30 IV CONCLUSIONS AND RECOMMENDATIONS... 34 4. Conclusions. 34 4. Recommendations.. 34 REFERENCES... 36 APPENDIX A MSE PERFORMANCE OF CLOCK OFFSET ESTIMATION BASED ON GMKPF... 39 APPENDIX B MSE PERFORMANCE OF CLOCK OFFSET AND SKEW ESTIMATION BASED ON GMKPF... 46 VITA... 58

vii LIST OF FIGURES FIGURE Page. Two-way message exchange mechanism which assumes only cloc offset... 8. Two-way message exchange mechanism assuming both cloc offset and sew..3 Dynamic state space model... 8.4 Dynamic state space model of cloc offset and sew... 7 3. Cloc offset state estimation via GMKPF... 9 3. Cloc offset and sew state estimation via GMKPF... 3 A. MSEs of cloc offset estimators for symmetric Gaussian delays [ =].. 39 A. MSEs of cloc offset estimators for symmetric exponential random delays [ =]...... 40 A.3 MSEs of cloc offset estimators for asymmetric Gaussian random delays [ =, =] 40 A.4 MSEs of cloc offset estimators for asymmetric exponential random delays [ =, =]. 4 A.5 MSEs of cloc offset estimators for Gamma random delays [( =, =) and ( =, =4)]... 4 A.6 MSEs of cloc offset estimators for Weibull random delays [( =, =) and ( =6, =)]... 4 A.7 MSEs of cloc offset estimators for a mixture of Gaussian [ =, =] and exponential [ =, =] random delays... 4

viii FIGURE Page A.8 MSEs of cloc offset estimators for a mixture of Gaussian [ =, =] and Gamma [( =, =) and ( =, =4)] random delays.... 43 A.9 MSEs of cloc offset estimators for a mixture of Gaussian [ =, =] and Weibull [( =, =) and ( =6, =)] random delays. 43 A.0 MSEs of cloc offset estimators for a mixture of exponential [ =, =] and Gamma [( =, =) and ( =, =4)] random delays 44 A. MSEs of cloc offset estimators for a mixture of exponential [ =, =] and Weibull [( =, =) and ( =6, =)] random delays 44 A. MSEs of cloc offset estimators for a mixture of Gamma [( =, =) and ( =, =4)] and Weibull [( =, =) and ( =6, =)] random delays. 45 B. MSEs of cloc offset and sew estimators for symmetric Gaussian delays [ =]... 46 B. MSEs of cloc offset and sew estimators for symmetric exponential random delays [ =]...... 47 B.3 MSEs of cloc offset and sew estimators for asymmetric Gaussian random delays [ =, =] 48 B.4 MSEs of cloc offset and sew estimators for asymmetric exponential random delays [ =, =]... 49 B.5 MSEs of cloc offset and sew estimators for Gamma random delays [( =, =) and ( =, =4)]... 50 B.6 MSEs of cloc offset and sew estimators for Weibull random delays [( =, =) and ( =6, =)]... 5 B.7 MSEs of cloc offset and sew estimators for a mixture of Gaussian [ =, =] and exponential [ =, =] random delays... 5

ix FIGURE Page B.8 MSEs of cloc offset and sew estimators for a mixture of Gaussian [ =, =] and Gamma [( =, =) and ( =, =4)] random delays...... 53 B.9 MSEs of cloc offset and sew estimators for a mixture of Gaussian [ =, =] and Weibull [( =, =) and ( =6, =)] random delays... 54 B.0 MSEs of cloc offset and sew estimators for a mixture of exponential [ =, =] and Gamma [( =, =) and ( =, =4)] random delays 55 B. MSEs of cloc offset and sew estimators for a mixture of exponential [ =, =] and Weibull [( =, =) and ( =6, =)] random delays... 56 B. MSEs of cloc offset and sew estimators for a mixture of Gamma [( =, =) and ( =, =4)] and Weibull [( =, =) and ( =6, =)] random delays. 57

CHAPTER I INTRODUCTION. Wireless Sensor Networs and Their Applications Advancement in Micro-Electro-Mechanical systems (MEMS), wireless communications and digital electronics has allowed the development of small, low-cost, energy efficient, and multi-functional wireless sensing devices. Wireless sensors are featured in general with environmental sensing and data processing capabilities, and with means to freely communicate over short distance. This enables the sensor nodes to efficiently collaborate for collecting and processing information and to effectively operate over a large region. A system composed of a large number of sensor nodes is called a Wireless Sensor Networ (WSN) [], []. Given the promising features of WSNs, the number of applications involving WSNs has grown rapidly, and as a result, WSNs have found a wide range of practical applications in health, military, environment, trading, security, etc. For example, in a hospital, sensor networs can remotely monitor the patient s physiological data and allow doctors to identify pre-determined symptoms in earlier stages. WSNs can relay the origin of wild fire by strategically, randomly, and densely deploying the sensor nodes in the forest. They can be embedded into home appliances such as TVs, refrigerators, and micro-waves and can be controlled locally and remotely by users. As a consequence, in the near future, WSNs could be integrated into many aspects of a person s daily life []. This thesis follows the style of IEEE Transactions on Automatic Control.

There are many factors which influence the design of WSNs such as nodes fault tolerance, scalability, production costs, operation environments, networ topology, hardware constraints, transmission media, and power consumption []. Especially, the sensor nodes power is limited because the scale of deployment maes them mostly inaccessible and their batteries are restricted in terms of power levels. In addition, data communication is the most vital operation in WSNs and requires huge portion of energy consumption which in general is greater than the energy required by the local data processing at each individual sensor node. Therefore, the most crucial factor in designing WSNs is to ensure the energy efficiency.. Importance of Robust Cloc Synchronization In any distributed systems, cloc synchronization is a critical piece of infrastructure because it is a procedure for providing a common nowledge of time across the entire system which allows collaboration among the nodes in the system. WSN is an example of such a distributed system that needs cloc synchronization in order to perform a number of fundamental operations such as data fusion, power management, transmission scheduling, tracing, etc. []. Every individual sensor node in a networ has its own cloc function defined by its cloc offset and sew parameters. Cloc synchronization between any two nodes is generally accomplished by message exchanges. Due to the presence of non-deterministic and possibly unbounded message delays, messages can be delayed arbitrarily while transferring messages between any two nodes. This maes the cloc synchronization a

3 difficult and complex problem. The most commonly proposed non-deterministic networ delay distributions are the Gaussian, Exponential, Gamma, and Weibull probability density functions (pdfs) [3], [4], [5], [6]. The maximum lielihood estimators (MLEs) of cloc offset and cloc sew have been proposed for the Gaussian delay model and Exponential Delay model in [3], [7], respectively. However, paper [8] shows that MLE for the Gaussian delay model (GMLE) and the MLE for the Exponential delay model (EMLE) are quite sensitive to the networ delay distribution. Consequently, any uncertainty in the nowledge of networ delay distribution increases the number of message errors and the number of message retransmissions, which cause WSNs to waste more power to achieve synchronization. Therefore, robust cloc synchronization techniques for WSNs are required to withstand the unnown or possibly time-varying distributions of the networ delays in the uplin and downlin of message exchanges. In any distributed system, cloc synchronization is a critical piece of infrastructure because it is a procedure for providing a common nowledge of time across the entire system which allows collaboration among nodes in the system. WSNs are one of such systems that need cloc synchronization in order to perform a number of fundamental operations such as data fusion, power management, transmission scheduling, tracing, etc []. Reference [4] proposes a robust cloc offset estimation method in the presence of non-gaussian random delays, referred to as the Gaussian Mixture Kalman Particle Filter (GMKPF). The Mean-Square Error (MSE) performance of GMKPF is superior than the MSE performance of GMLE and EMLE in general networ delay distributions and in the presence of a small number of message exchanges. A reduced number of message

4 exchanges helps WSNs consume less energy which is one of the most important points in designing WSNs. This thesis proposes to study a joint cloc offset and sew estimation method based on GMKPF. The MSE performance of GMKPF, GMLE, and EMLE is simulated under Gaussian, Exponential, Gamma, Weibull delay distributions, and a mixing of two delay distributions. The computer simulation results show that the proposed scheme has superior performance relative to GMLE and EMLE in most networ delay distributions. Therefore, GMKPF represents a very reliable scheme for joint cloc offset and sew estimation. This will help guarantee long-term reliability of cloc synchronization and reduce networ-wide energy consumption, which is one of the ey strategies for the successful deployment of long-lived WSNs..3 Literature Review A few protocols have been proposed for cloc synchronization of WSNs. This research analyzes the cloc sync protocols relying on two-way message exchanges between two nodes. The adopted scenario is similar to the Timing synch Protocol for Sensor Networs (TPSN) [9]. TPSN acts as a conventional sender-receiver protocol which assumes two operational stages: the level discovery phase and the synchronization phase. The global synchronization of the networ is achieved by the two-way message exchanging mechanism through adjusting only its cloc offset. By applying a probabilistic approach to deal with the cloc synchronization problem in WSNs, Abdel-Ghaffar [3] classified the lin delay as deterministic and nondeterministic components. Abdel-Ghaffar also reviews five different cloc offset

5 estimation algorithms which include the median round delay, the minimum round delay, the median phase, and the average phase under symmetric exponential lin delay. Later, Jese [0] mathematically proved that MLE of cloc offset exists when the fixed delay components in each direction are equal and the variable delay is exponentially distribution with an unnown mean. This result was also consistent with the previously proposed estimators. In a wor of Noh et al. [8], MLE for the symmetric Gaussian delay model was derived and the Joint Maximum Lielihood Estimator (JMLE) of cloc offset and cloc sew for the symmetric Gaussian model delay was also derived. In addition, Noh et al. [8] also proposed a less computationally complex method for cloc offset and sew estimation under Gaussian and exponential delays, called the Maximum Lielihood-Lie Estimator (MLLE). Furthermore, JMLE of cloc offset and sew estimation for the symmetric exponential model delay was proposed by Chaudhari []. However, the previously proposed estimation methods are very sensitive to changes in the networ delay distribution. Recently, Kim et al. [4] developed a robust estimation scheme for cloc offset called the Gaussian Mixture Kalman Particle Filtering (GMKPF), and which was shown to provide better performance for arbitrary networ delay distributions relative to MLE. Drawbacs of the study undertaen by Kim et al. [4] are the facts that analytical closed form expressions for MSE do not necessarily exist and that lower bounds are hard to derive. While significant progress has been made in the effort to efficiently estimate cloc offset and sew, estimation schemes which are not sensitive to non-gaussian/nonexponential networ delays have not been developed yet. This research, therefore, will

6 conduct a series of computer simulation to assess the performance of GMKPF in estimating both the cloc offset and sew parameters..4 Problem Description The cloc offset and sew parameters are two factors that determine the accuracy and guarantee long-term reliability of time synchronization in WSNs, and therefore, reduction of networ energy consumption which is the ey strategy in obtaining energy efficiency usage. Cloc synchronization between any two nodes is generally accomplished by message exchanges. Due to the presence of non-deterministic message delays, messages can get delay arbitrarily. It is important to develop an estimator which performs well in any networ delay environment. As depicted later in Chapter II, in the Figure on page, the cloc offset and sew measurement is modeled as equation on page. In order to obtain a robust estimation scheme for cloc offset and sew, GMKPF was first studied and the MSE performance of GMKPF was compared with the MSE performance of GMLE and EMLE through computer simulations. Then, GMKPF was used to estimate the cloc offset and sew parameters by formulating the problem as described in Section.7. A series of computer simulations was conducted in order to verify the MSE performance of GMKPF through comparisons with JMLE and MLLE.

7.5 Research Objective This research aims at two main objectives: Study the previously proposed cloc offset and joint cloc offset and sew parameter estimation methods: MLE, JMLE, MLLE, and GMKPF. Conduct a computer simulation to study the robustness of the cloc offset and sew estimation scheme based on GMKPF assuming Gaussian and general non- Gaussian delay distributions such as exponential, Gamma, and Weibull and compare their MSE performance with that of JMLE and MLLE. By achieving the above goals, this research was able to provide a robust cloc synchronization method by estimating both the cloc offset and sew parameters.

8 CHAPTER II SYSTEM DESCRIPTION AND BACKGROUND THEORIES. Two-way Message Exchange Mechanism Without Cloc Sew Assuming no cloc sew, a number of N two-way message exchanges or senderreceiver synchronization (SRS) exchanges are graphically shown in Figure.. T, and T 4, represent the timestamps measured by the local cloc of node A, while T, and T 3, represent the timestamps measured by the local cloc of node B at the th message exchange. Figure.: Two-way message exchange mechanism which assumes only cloc offset Based on the above pairwise message exchange mechanism, the cloc offset measurement model can be represented in terms of the following two equations [8]: T T d X,, T T d Y, 4, 3,, (.) where d denotes the fixed portions of the delay, and represents the cloc offset between node A and node B. X and Y denote the variable portions of delay.

9 The equation (.) can also be rewritten as U d X V d Y,, (.) where the time differences U and V corresponding to the th uplin and downlin message exchange, are defined as U T, T, and V T4, T3, respectively.. Maximum Lielihood Cloc Offset Estimation Delay measurements in equation (.) produce a MLE of the cloc offset when the fixed delays in each direction are equal and the variable delays in each direction assume the same distribution, namely Gaussian or exponential distribution... Gaussian Delay Model Assuming the symmetric Gaussian delay for uplin and downlin, the set of delay observations { X } N and { Y } N are independent and normally distributed with the same mean and variance. The lielihood function based on the observations { U } N and { V } N is given by [8] L(,, ) ( ) exp{ [ ( U d ) ( V d ) ]}. N N N (.3) Differentiating the log-lielihood function gives N ln L( ) [ ( U V )]. (.4)

0 Hence, the MLE of cloc offset is given by ˆ N ln L( GMLE ) [ ˆ ( )] 0 ˆ GMLE U V GMLE N ( U V ) U V N ˆ GMLE. (.5).. Exponential Delay Model Assuming the symmetric exponential delay model for the uplin and downlin, the set of delay observations { X } N and { Y } N are independent and exponentially distributed with the same mean. { U } N and { V } N define the order statistics of the sequences of observations { U } N and{ V } N, respectively. The lielihood function based on the observations is given by [9] N N N (,, ) exp{ [ ]} IU [ () d, V () d]. L d U V Nd (.6) MLE of cloc offset is given by [9] U V (.7) () () ˆ EMLE..3 Two-way Message Exchange Mechanism with Cloc Offset and Sew Due to the imperfections of the cloc oscillator, the cloc of each node presents different cloc offsets and sews. Moreover, the cloc offset between two nodes actually eeps increasing because of the effect of cloc sew. Therefore, a fixed value model or

equation (.) for cloc offset is not sufficient for practical situations. As a consequence, estimating the cloc offset and sew will increase the accuracy and maintain long-term reliability of synchronization. Figure.: Two-way message exchange mechanism assuming both cloc offset and sew. The effect of cloc offset and sew are depicted in Figure.. Here, the timestamps in the th message exchange T, and T 4, are measured at node A, and T, and T 3, are measured at node B. Assuming T, is set to zero and it serves as the reference time, the cloc offset and sew measurement could be modeled as [] T ( T d X ),,, T ( T d Y ), 3, 4, (.8) where represents the cloc sew parameter..4 Joint Maximum Lielihood Estimation (JMLE) of Cloc Offset and Sew The theory applied thus far for finding the MLE for the cloc offset (assuming no cloc sew) can be extended to find JMLE for a more general cloc model (.8).

.4. Gaussian Delay Model Assume { X } N and { Y } N variables with variance are zero mean independent Gaussian distributed random and the fixed portion of delay d is nown. The JMLE of cloc offset and sew based on the observations { T, } N, { T, } N, { T3, } N, and { T4, } N is given by [] N N N, 4,, 3,, 3, GMLE N N ( T T ) ( T T ) ( T T ) Q ˆ, ( T T ) ( T T ) NQ, 3,, 4, (.9) N[ ( T T ) ( T T ) ( T T ) Q], 4,, 3,, 3, ˆ GMLE N N N, 4,, 3,, 4, N N N N N ( T T )[ ( T T ) ( T T ) NQ] ( T, T3, ), ( T T ), 4, (.0) N where,, 3, 4,, 3, Q [ T T T T ( T T ) d]..4. Exponential Delay Model Assuming the symmetric exponential delay model for the uplin and downlin then the set of delay observations { X } N and { Y } N are independently and exponentially distributed with the same mean. In paper [], the authors assumed that is nown and categorized JMLE of cloc offset and sew into four cases:

3 Case I: d nown, nown; Case II: d unnown, nown; Case III: d nown, unnown; Case IV: d unnown, unnown. This thesis studies the MSE performance in the case that the fixed portion delay, d is nown, unnown and unnown. The algorithm for finding ˆ and ˆ when d is nown is given by the following operations: l, 3, l,, 4, l. Find ; l, T ( T d) T ( T d) ( T d) ( T d), 4, l T, T3, l ; ( T d) ( T d), 4, l,..., N and l,..., N; (.) l, l, l T,, r l T, 3, r. ( i, j) {(, l) and }; T d T d r,..., N;, r 4, r l, l, l T,, r l T, 3, r 3. ( m, n) {(, l) (, l) ( i, j), and }; T d T d r,..., N;, r 4, r (.) (.3) ˆ ˆ (.4) i, j m, n i, j m, n 4. EMLE min{, }; EMLE max{, };.5 Maximum Lielihood-Lie Estimator (MLLE) Figure. shows that the cloc difference between two nodes is monotonically increasing based on the linear cloc sew model in equation (.8). The largest cloc difference is between the first and last time stamps. From this intuition, MLLE is

4 proposed in [8] based on the information provided solely by the first and the last timestamps. Define the distances of the first and last time stamps as D() T, N T,, D() T, N T,, D(3) T3, N T3,, and D(4) T4, N T4,. From (.8), subtracting T, from T,N and subtracting T4, from T 4,N yields T T ( T T X X ),, N,, N, N T T ( T T Y Y ), 3, N 3, 4, N 4, N which can be further rewritten as D ( D X X ), () () N D ( D ( Y Y)). (3) (4) N (.5).5. Gaussian Delay Model In the Gaussian delay model, X, X N, Y, and Y N are independent and identically normally distributed random variables with zero mean and variance. Therefore, X N - X and Y N - Y are normally distributed random variables with zero mean and variance. Define X N - X as the variable P and Y N - Y as the variable R. Then the joint pdf of P and R is given by f pr p r 4 4 PR, (, ) exp[ ( )]. Thus, the lielihood function based on model (.5) is

5 D() D (4) L(, ) exp D() ( ) D(3) ( ). 4 4 D() D(3) Define ' and differentiating the log-lielihood function with respect to ' yields ln L( ', ) () (4) D() D(3) ' D() D(3) () () (3) (4) D() D(3) ( ' D ) ( ' D ), ln L( ˆ ', ) D () D (4) 0 ˆ ˆ D() ( ' ) D(3) ( ' ), ˆ ' D() D(3) D D D D ˆ '. Thus, the MLLE for the Gaussian delay model (GMLLE) is given by: D() D(3) ˆ GMLLE. D D D D () () (3) (4) (.6) Then, we could express the cloc offset estimator as follows: ' ' where U T ˆ T and V ˆ T T., GMLLE, ' ' ˆ U V GMLLE, (.7) GMLLE 4, 3,.5. Exponential Delay Model In the exponential delay model, X, X N, Y, and Y N are independent and identically distributed normal random variables with the same mean, and X N - X and Y N - Y are normally distributed random variables with zero mean. Similarly to the Gaussian delay model, X N - X is defined as the variable P and Y N - Y is defined as the variable R. P

6 and R are zero mean Laplace distributed random variables with variance and the joint pdf of P and R is given by fpr, ( pr, ) exp[ ( p r )]. The lielihood function based on model (.5) is D() D(3) L(, ) exp D() D(4). Substituting ', the lielihood function can be rewritten as L ( ', ) exp () ' () (4) (3) '. D D D D The maximization of the lielihood function is reduced to ˆ ' argmin( D ' D D D ' ). (.8) ' () () (4) (3) Therefore, the MLLE of the exponential delay model (EMLLE) is given by; (3) ˆ EMLLE, D(4) D(), D() D(3) D() D D() D(3) D() D(3), D() D(3). D() D (4) (.9) Then, we could express the cloc offset estimator as ˆ min U min V, (.0) ' ' N N EMLLE ' ' where U T ˆ T and V ˆ T T., EMLLE, EMLLE 4, 3,

7.6 Cloc Offset Estimation Based on the Gaussian Mixture Kalman Particle Filter Gaussian Mixture Kalman Particle Filter (GMKPF) provides a scheme which is robust to arbitrary random delay distributions such as asymmetric Gaussian, asymmetric exponential, Gamma, Weibull, as well as to mixtures of these delay models for estimating the cloc offset in WSNs. In addition, an advantage of GMKPF is in terms of its superior performance as compared to GMLE and EMLE..6. Problem Modeling In this section, the two-way message exchange mechanism is represented by a set of state-space and observation equations, called a dynamic state-space model (DSSM), as depicted in Figure.3. The observations y are conditionally independent given the state and are generated according to the probability density p( y x ). The state x evolves over time as an unobserved first order Marov process according to the probability density p( x x ). The DSSM can be described via the set of equations: x f ( x, v ) ( process equation), (.) y h( x, n ) ( observation equation), (.) where v is the process noise of state transition function f, and n is the observation noise of observation function h.

8 Figure.3: Dynamic state space model. Based on the cloc synchronization model (.) from Section., the unnown cloc offset s behavior follows a Gauss-Marov dynamic channel model of the form: (.3) A v, where A represents the state transition matrix for the cloc offset. The noise vector v is T modeled as a Gaussian random variable with zero mean and covariance E[ vv ] Q. The uplin delay U and downlin delay V of the th timing message exchange can be observed at node A and node B. The vector of observations z [ U V ] T can be expressed as in the observation equation (.), U d X z d n, V d Y (.4) where the observation noise vector is n [ ] T X Y. X and Y may assume any distribution such as Gaussian, exponential, Gamma, Weibull or a mixture of two distributions.

9 Given all the observation samples Z { z, z,, z}, the goal is to find the minimum MSE estimator of the unnown cloc offset, which is given by ˆ E[ Z ] p( Z ) d. The optimal method to recursively update the posterior density as new observations arrive is given by the recursive Bayesian estimation algorithm p( Z ) Cp( z ) p( Z ), where p( Z ) p( ) p( Z ) d and C p( z ) p( Z ) d. However, the expression of p( x Z ) can be obtained in closed-form expression only for the case of a linear Gaussian model. In equation (.4), X and Y could be non- Gaussian distributions, consequently, p( x Z ) cannot be analytically obtained. Alternatively, p( x Z ) can be recursively approximated via particle filtering..6. Framewor of GMKPF Particle filtering is a sequential Monte Carlo methodology [] where the basic idea is the recursive computation of relevant probability distributions using the concepts of importance sampling (IS) and approximation of probability distributions with discrete random measures. GMKPF in [4] is developed based on the Gaussian mixture sigma point particle filter (GMSPPF) proposed in [3]. The GMSPPF combines an IS based measurement update step with a Sigma-Point Kalman Filter (SPKF) [4] based on the

0 Gaussian sum filter [5] for the time update and proposal density function. Since the DSSM of cloc synchronization explained in the previous section is a linear model, GMKPF implements the Kalman Filter (KF) [6] instead of SPKF. The framewor of GMKPF mainly relies on these three elements: ) Gaussian mixture model (GMM) approximation Any probability px ( ) can be approximated as closely as desired by a Gaussian mixture model (GMM) of the following form: G ( g) ( g) ( g) ( ) g ( ) ( ;, ), g px p x Nx P (.5) ( g ) where G stands for the number of mixing components, are the mixing weights and N( x;, P) denotes the normal distribution with mean vector and positive definite covariance P. For example, the following density function can be approximated by GMMs: G'' ( g'') ( g'') ( g''), ( ) g( ) ( ;, ). g '' posterior density p x Z p x Z N x P The predicted and updated Gaussian component mean and covariance of pg ( x Z) are calculated using KF. ) Importance sampling (IS) based measurement update In particle filtering, the distributions are approximated by discrete random measures defined by particles and weights assigned to the particles. If the distribution of interest is pxand ( ) its approximating random measure is x w, where x ( m) are the ( m) ( m) M {, } m

particles, w ( m) are the corresponding weights, and M denotes the number of particles used in the approximation, then px ( ) is approximated by [] M ( m) ( m) ( ) ( ), px w x x m where () is the Dirac delta function. With this approximation, computations of expectations are simplified and approximated by [] M ( m) ( m) ( ( )) ( ). E g x w g x m Based on DSSM model (.3) and (.4), the conditional mean state and corresponding error covariance are calculated as follows: M ( m) ( m) w m, (.6) M ( m) ( m) ( m) T m P w [ ][ ]. (.7) The updated importance weights can be obtained by [3] w pz ( ) p( ). ( m) ( m) ( m) ( m) ( m) w ( m) ( m) ( 0:, z0: ) The distribution (, z ) is nown as an importance function. In GMKPF, ( m) ( m) 0: 0: the proposal distribution for (, z ) is approximated by GMM from the ( m) ( m) 0: 0: ban of KFs. By sampling the particles from such a distribution will move the particles to areas of high lielihood which in turn resolves the sample depletion problem of particle filtering. Moreover, using the GMM approximation on the predictive posterior density function preserves the ernel smoothing nature.

3) Weighted EM for resampling and GMM recovery In standard particle filtering, resampling is needed in order to eep good performance of particle filter. Resampling is a scheme that discards particles which are assigned negligible weights and replicates particles with large weights []. Unfortunately, resampling causes particle depletion in cases where the measurement lielihood is very high. Consequently, the set of particles will be only copies of the same particle [7]. In GMKPF, the posterior density is represented by GMM; hence, the standard resampling method can be replaced by a weighted Expectation-Maximization (EM) algorithm [8]. It directly recovers a maximum-lielihood G-component GMM fit to the set of weighted samples; as a result, it prevents the particle depletion problem. The EM algorithm provides an iterative method of estimating via arg max pz ( ). The G-component GMM is specified by the parameter set. () ( ) () ( ) () ( ) {,..., G,,..., G,,..., G P P } The EM algorithm represents a two-step iterative algorithm which is described by the following two steps: Estep Q E p z Mstep ( j) ( j) : ( ) [log ( ) ], ( j) ( j) : argmax Q( ). Finally, the conditional mean state estimate and corresponding error covariance are calculated as follows:

3 G ( g) ( g) g, G ( g) ( g) ( g) ( g) T P g P ( )( )..6.3 GMKPF Algorithm In this section, the full GMKPF algorithm will be presented based on the framewor described in the previous section. ) Time update and proposal distribution generation.) At time, initialize the state densities: The posterior state density is approximated by G ( g) ( g) ( g) g g p ( Z ) N( ;, P ); The process noise density is approximated by I () i () i () i g v v i p ( v ) N( v ;, P ); The observation noise density is approximated by J ( j) ( j) ( j) g( ) ( ; n, ) n j p n N n P..) During the time update step of KF (employing the system process equation (.3), posterior state density pg( Z ) and process noise density pg( v ) from above), a Gaussian approximated predictive state

4 G' ( g') ( g') ( ') density ˆ ( ˆ ˆ g pg Z ) N( ;, P ) is calculated. The Gaussian g ' approximated posterior state density, G'' ( g'') ( g'') ( g'') pˆ g( Z) N( ;, P ) g '', ) Measurement update is calculated during the measurement update step of KF (employing the system observation equation (.4), current observation z pre-predictive state density pˆ g( Z ) and observation noise density pg ( n ))..) Draw M samples ( m) { ;,..., } m M from the importance density function pˆ ( Z ). g.) Calculate the corresponding importance weights: pz pˆ Z w. ( m) ( m) ( m) ( ) g( ) ( m) pˆ g( Z).3) Normalize the weights w ( m) w ( m) M ( m) w m.4) Use a weighted EM algorithm to fit a G-component GMM to the set of. weighted particles w m M, which represent the update ( m) ( m) {, ;,..., } GMM approximate state posterior distribution at time, p ( Z ). 3) Inference of the conditional mean and covariance g

5 3.) M ( m) ( m) w m and M ( m) ( m) ( m) T [ ][ ] m P w or equivalently, upon fitting the GMM approximate posterior distribution through the weighted EM algorithm, calculate the conditional mean state estimate and corresponding error covariance..7 Joint Cloc Offset and Sew Estimation Based on GMKPF In this section, a robust scheme for estimation of cloc offset and sew in wireless sensor networs is designed based on GMKPF. The only difference is the process equation and observation equation of state space model which will be described in Section.7...7. Problem Modeling Consider the cloc offset and sew model (.8); described by the equation T ( T d X ),,, T ( T d Y ). 3, 4, Let B be defined as. Then, the model above can be rewritten as; T T U ( dx )( ) T,,, B B, T T V ( dy )( ) T. 4, 3, B B 4, (.8) Assuming, the unnown and B obey a Gauss-Marov dynamic channel model of the form:

6 A v B, B, x Ax v (.9) where A represents the state transition matrix and x [, ] T B. The noise vector v T is modeled as a Gaussian random variable with zero mean and covariance Evv [ ] Q. Based on observation model (.8), the observation equation can be written as z U ( )(, ) d X B B, T, V ( d Y )( ) T B, B, 4, B, 0 B, C d ( B, ) n, 0 B, B, (.30) where the observation noise vector n [ ] T X Y. X and Y may assume any distribution such as Gaussian, exponential, Gamma, Weibull or a mixture of two distributions. C [ T, T4, ] T is treated as exogenous input to the state observation function. In short, we can write equation (.30) as z h(, B,, C) where h() is the state observation function. In Figure.4 is depicted a DSSM model for cloc offset and sew estimation method based on GMKPF.

Figure.4: Dynamic state space model of cloc offset and sew. 7

8 CHAPTER III PERFORMANCE OF GMKPF 3. Cloc Offset Estimation Based on GMKPF In this section, computer simulation results will be presented to determine the performance of GMKPF (Section.6), GMLE and EMLE (Section.) methods for estimating the cloc offset in WSNs. The observation noise delay models are considered to be twelve delay distributions: symmetric Gaussian, symmetric exponential, asymmetric Gaussian, asymmetric exponential, Gamma, Weibull and mixtures of Gaussian and exponential, Gaussian and Gamma, Gaussian and Weibull, exponential and Gamma, exponential and Weibull, and Gamma and Weibull. The number of Monte Carlo simulations is 000. The state transition matrix A of process equation is 0.99999. The process noise v is assumed to be Gaussian with constant covariance, Q = e-5. The number of particles and GMM components are 000 and 5, respectively. The experiment was done using Matlab and ReBEL Toolit. Figure 3. shows a plot of the cloc offset estimation in each Monte Carlo simulation. The MSE performances of estimators are averaged over 000 Monte Carlo runs. Figures A.-A. show the MSEs of the estimators assuming that the random delay models are symmetric Gaussian, exponential, asymmetric Gaussian, exponential, Gamma, and Weibull distributions, and mixtures of two distributions respectively. ReBEL is a Matlab toolit for sequential Bayesian inference in General DSSMs. ReBEL is developed by MLSP Group at OGI and can be freely downloaded from http://cslu.ece.ogi.edu/mslp/rebel for academic and /or non-commercial use.

9 Figure 3.: Cloc offset state estimation via GMKPF. The subscripts and are used to differentiate between the parameters of delay distributions corresponding to the uplin and the downlin, respectively. For example, in Figure A.3, the parameters and are standard deviations of uplin and downlin assuming Gaussian delay distributions, respectively. The number of observations which are experimented in the simulation are 0, 5, 0, 5 and 30. To construct a mixing noise delay distribution, two distributions are mixed with equal weight. For example, a Gaussian delay and exponential delay are generated and mixed with an equal weight of 50%. The number of observations which are experimented for the mixing case is 0, 6,, and 8, respectively. From Figures A.-A., GMKPF clearly outperforms GMLE and EMLE, and results in reduction of MSE over 00%. GMLE performs better than EMLE in the presence of Gaussian random delays (Figures A., A.3, A.7 and A.9) but it performs

30 poorer than EMLE in the presence of exponential, Gamma and Weibull random delays (Figures A., A.4-A.6, A.8, and A.0-A.). This happens because Gamma and Weibull random delays are closer to the exponential random delay. 3. Combination of Cloc Offset and Sew Estimation Based on GMKPF In this section, computer simulation results will be presented to determine the performance of GMKPF (Section.7), JMLE for Gaussian and exponential delay models (Section.4), and MLLE for Gaussian and exponential delay models (Section.5) for estimating the cloc offset and sew in WSNs. The observation noise delay model is the same as in Section 3.. The state transition matrix A of process equation is 0.99999. The process noise v is assumed to be Gaussian with constant diagonal matrix covariance: Q = e 5 0 0 e 5. The number of particles and GMM components are 000 and 5, respectively. The experiment was done using Matlab and ReBEL Toolit. Figure 3. shows a plot of the cloc offset and cloc sew estimation for each Monte Carlo simulation. The MSE performances of estimators are averaged over,000 Monte Carlo runs. Figures B.-B.6 show the MSE of the estimators assuming that the random delays assume symmetric Gaussian, exponential, asymmetric Gaussian, exponential, Gamma, and Weibull distributions, respectively. The number of observations conducted in this simulation are 0, 5, 0, 5 and 30, respectively.

3 Figure 3.: Cloc offset and sew state estimation via GMKPF. In the symmetric Gaussian delay case, Figure B., GMKPF and GMLE give almost the same performance in estimating the cloc sew. However, GMKPF performs

3 better in estimating the cloc offset. Notice that EMLE always give as a value: Not a Number (NaN) for any number of observations. This happens because the possible cloc sew parameters (.) are not in the valid region (see equations (.) and (.3)). MLLE performs poorer than MLE and GMKPF in estimating jointly the cloc offset and sew. In the symmetric exponential delay case, Figure B., GMKPF gives slightly poorer MSE performance in estimating cloc sew when compared to EMLE. GMKPF gives better MSE performance in estimating cloc offset than EMLE, GMLE, GMLLE, and EMLLE. GMLE performs poorer than GMKPF and EMLE for joint estimation of cloc offset and sew estimation. Both GMLLE and EMLLE give relatively the same performance and are poorer than MLE and GMKPF for cloc offset and cloc sew estimation. In other cases, Figures B.3-B.6, GMKPF performs better than MLEs and MLLEs in estimating both the cloc offset and sew. Note that EMLE performs better GMLE in the exponential delay case but EMLE is very sensitive to other delay models, especially the Gaussian delay. It is interesting to note that the MSE of GMLE exhibits better performance than MLLE in estimating the cloc sew parameter for any delay distributions. GMLE gives poorer performance in estimating the cloc offset for Gamma and Weibull delay models which are closer to the exponential distribution than the Gaussian distribution. In addition, EMLLE and GMLLE give almost the same performance in estimating the cloc sew parameter regardless of the type of random delays. This is due to the fact that the performance of

33 4 the MLLE is dominated by the set of distances D() i, which do not vary much with respect to the type of random delays. To quantify the robustness of the estimators further, Figures B.7-B. show the MSE performance of the GMKPF, GMLE, EMLE, GMLLE, and EMLLE under a mixture of two networ delay distributions. The numbers of observations which are experimented are 0, 6,, and 8, respectively. GMKPF clearly outperforms the other estimators in every case. GMLE gives better performance than EMLE, GMLLE and EMLLE if the networ delay model is closer to a Gaussian (Figures B.7-B.9). EMLE gives a NaN value in the case that the networ delay is a mixture of Gaussian and other delay models (Figures B.7-B.9) and performs better than GMLE and MLLEs in case that the networ delay is closer to the exponential delay model (Figures B.0-B.). GMLLE and EMLLE give almost the same performance for estimating the cloc sew parameter. However, when they estimate the cloc offset, the MSE performance diverges (Figures B.7-B.). This shows that MLLEs are not robust to a mixing networ delay model. i

34 CHAPTER IV CONCLUSIONS AND RECOMMENDATIONS 4. Conclusions A joint cloc offset and sew estimation method was developed based on GMKPF and its robustness tested via computer simulations. The final results are: GMKPF yields the best MSE performance relative to the most commonly proposed estimators in the presence of non-deterministic networ delays distributions: Gaussian, exponential, Gamma, and Weibull, and mixture between two of these distributions. The main drawbac of GMKPF is its computational complexity. In addition, it taes considerable time in terms of calculation. However, GMKPF could yield the same MSE performance as MLE and MLLE in the case of lower number of message exchanges. 4. Recommendations GMKPF s performance depends on the number of particles and the number of Gaussian mixture components. Although, increasing the number of particles and GMM components lead to better performance, it requires a longer time in terms of completing the required computations. GMKPF could be studied further to identify the best number of particles and GMM components in estimating the cloc offset and sew. GMKPF s performance is also very sensitive to the process state transition matrix A (.9), process

35 noise and initialisation of GMKPF estimator. The performance can vary drastically due to changes in these parameters. Thus, GMKPF could give even better performance, provided there is a study to determine the optimal values of these parameters.

36 REFERENCES [] E. Serpedin and Q. M. Chaudhari, Synchronization in Wireless Sensor Networs : Parameter Estimation, Performance Benchmars and Protocols. New Yor: Cambridge University Press, 009. [] I. F. Ayildiz, W. Su, Y. Sanarasubramaniam, and E. Cayirci, "Wireless sensor networs: a survey," Computer Networs, vol. 38, no. 4, pp. 393-4, Mar 5, 00. [3] H. S. Abdel-Ghaffar, "Analysis of synchronization algorithms with time-out control over networs with exponentially symmetric delays,", IEEE Trans. on Communications, vol. 50, no. 0, pp. 65-66, 00. [4] J. Kim, J. Lee, E. Serpedin, and K. Qaraqe, "A robust estimation scheme for cloc phase offsets in wireless sensor networs in the presence of non-gaussian random delays," Signal Processing, vol. 89, no. 6, pp. 55-6, 009. [5] A. Leon-Garcia, Probability and Random Processes for Electrical Engineering, nd ed. Reading, MA: Addison-Wesley, 994. [6] A. Papoulis, Probability, Random Variables, and Stochastic Processes. New Yor: McGraw-Hill, 00. [7] J. Elson, L. Girod, and D. Estrin, "Fine-grained networ time synchronization using reference broadcasts," in Proc. of the Fifth Symposium on Operating System Design and Implementation, Boston, MA, Dec. 00.

37 [8] K.-L. Noh, Q. M. Chaudhari, E. Serpedin, and B. W. Suter, "Novel cloc phase offset and sew estimation using two-way timing message exchanges for wireless sensor networs," IEEE Trans. on Communications, vol. 55, no. 4, pp. 766-777, April 007. [9] S. Ganeriwal, R. Kumar and M. B. Srivastava, "Timing-sync protocol for sensor networs," in Proc. First Int. Conf. Embedded Networ Sensor Systems, pp. 38-49, 003. [0] D. R. Jese, "On maximum-lielihood estimation of cloc offset," IEEE Trans.s on Communications, vol. 53, no., pp. 53-54, 005. [] Q. M. Chaudhari, E. Serpedin, and K. Qaraqe, "On maximum lielihood estimation of cloc offset and sew in networs with exponential delays," IEEE Trans. on Signal Processing, vol. 56, no. 4, pp. 685-697, 008. [] P. M. Djuric, J. H. Kotecha, J. Zhang, Y. Huang, and T. Ghirmai, "Particle filtering," IEEE Signal Processing Magazine, vol. 0, pp. 9-38, 003. [3] R. van der Merwe and E. Wan, "Gaussian mixture sigma-point particle filters for sequential probabilistic inference in dynamic state-space models," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, vol. 6, pp. VI-70-4, 003. [4] R. van der Merwe and E. Wan, "Sigma-point alman filters for probabilistic inference in dynamic state-space models," in Proc. Worshop Advances in Machine Learning, Montreal, Canada, Jun 003.

38 [5] D. Alspach and H. Sorenson, "Nonlinear Bayesian estimation using Gaussian sum approximations," IEEE Trans. on Automatic Control vol. 7, pp. 439-448, 97. [6] S. M. Kay, Fundamentals of Statistical Signal Processing. Englewood Cliffs, NJ: PTR Prentice-Hall, 993. [7] A. Doucet, N. de Freitas, and N. Gordon, Sequential Monte Carlo Methods in Practice. New Yor: Springer, 00. [8] G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions, nd ed. Hoboen, NJ: Wiley Interscience, 008.

39 APPENDIX A MSE PERFORMANCE OF CLOCK OFFSET ESTIMATION BASED ON GMKPF Figure A.: MSEs of cloc offset estimators for symmetric Gaussian delays [ =].

40 Figure A.: MSEs of cloc offset estimators for symmetric exponential random delays [ =]. Figure A.3: MSEs of cloc offset estimators for asymmetric Gaussian random delays [ =, =].

4 Figure A.4: MSEs of cloc offset estimators for asymmetric exponential random delays [ =, =]. Figure A.5: MSEs of cloc offset estimators for Gamma random delays [( =, =) and ( =, =4)].

4 Figure A.6: MSEs of cloc offset estimators for Weibull random delays [( =, =) and ( =6, =)]. Figure A.7: MSEs of cloc offset estimators for a mixture of Gaussian [ =, =] and exponential [ =, =] random delays.

43 Figure A.8: MSEs of cloc offset estimators for a mixture of Gaussian [ =, =] and Gamma [( =, =) and ( =, =4)] random delays. Figure A.9: MSEs of cloc offset estimators for a mixture of Gaussian [ =, =] and Weibull [( =, =) and ( =6, =)] random delays.

44 Figure A.0: MSEs of cloc offset estimators for a mixture of exponential =, =] and Gamma [( =, =) and ( =, =4)] random delays. [ Figure A.: MSEs of cloc offset estimators for a mixture of exponential =, =] and Weibull [( =, =) and ( =6, =)] random delays. [

45 Figure A.: MSEs of cloc offset estimators for a mixture of Gamma [( =, =) and ( =, =4)] and Weibull [( =, =) and ( =6, =)] random delays.

46 APPENDIX B MSE PERFORMANCE OF CLOCK OFFSET AND SKEW ESTIMATION BASED ON GMKPF Figure B.: MSEs of cloc offset and sew estimators for symmetric Gaussian random delays [ =].

Figure B.: MSEs of cloc offset and sew estimators for symmetric exponential random delays [ =]. 47

48 Figure B.3: MSEs of cloc offset and sew estimators for asymmetric Gaussian random delays [ =, =].

49 Figure B.4: MSEs of cloc offset and sew estimators for asymmetric exponential random delays [ =, =].

50 Figure B.5: MSEs of cloc offset and sew estimators for Gamma random delays [( =, =) and ( =, =4)].

5 Figure B.6: MSEs of cloc offset and sew estimators for Weibull random delays [( =, =) and ( =6, =)].

5 Figure B.7: MSEs of cloc offset and sew estimators for a mixture of Gaussian [ =, =] and exponential [ =, =] random delays.

53 Figure B.8: MSEs of cloc offset and sew estimators for a mixture of Gaussian [ =, =] and Gamma [( =, =) and ( =, =4)] random delays.

54 Figure B.9: MSEs of cloc offset and sew estimators for a mixture of Gaussian [ =, =] and Weibull [( =, =) and ( =6, =)] random delays.

55 Figure B.0: MSEs of cloc offset and sew estimators for a mixture of exponential [ =, =] and Gamma [( =, =) and ( =, =4)] random delays.

56 Figure B.: MSEs of cloc offset and sew estimators for a mixture of exponential [ =, =] and Weibull [( =, =) and ( =6, =)] random delays.

57 Figure B.: MSEs of cloc offset and sew estimators for a mixture of Gamma [( =, =) and ( =, =4)] and Weibull [( =, =) and ( =6, =)] random delays.

58 VITA Name: Address: Education: Sawin Saibua Department of Electrical and Computer Engineering, Texas A&M University, 4 Zachry Engineering Center, TAMU 38 College Station, Texas 77843-38. B.Eng., Electrical Engineering, Chulalongorn University, Bango, Thailand, 008. M.S., Electrical Engineering, Texas A&M University, College Station, 00.