Maximum Likelihood Diffusive Source Localization Based on Binary Observations

Transcription

1 Maximum Lielihood Diffusive Source Localization Based on Binary Observations Yoav Levinboo and an F. Wong Wireless Information Networing Group, University of Florida Gainesville, Florida , USA Abstract In this paper, we construct the maximum lielihood (ML estimator of diffusive source location based on binary observations. We utilize two different estimation approaches, ML estimation based on all the observations (i.e., batch processing and approximated ML estimation using only new observations and the previous estimate (i.e., real time processing. he performance of these estimators are compared with theoretical bounds and are shown to achieve excellent performance. I. Introduction We investigate the problem of estimating the location of an instantaneous source of an arbitrary gas using a simple chemical sensor networ. We assume that the gas spreads by diffusion 1 and the sensors mae binary decisions on the existence of the gas by measuring its concentration in their immediate vicinity. he sensor outputs are periodically relayed to a fusion center that uses this information to estimate the location of the source. It turns out from the analysis of the Cramer-Rao bound (CRB 2 that if we tae the physical transport mechanism of the gas into account, we may achieve excellent performance in the estimation of the diffusive source location despite of the observations being binary. he only condition for this to be true is that the measurements prior to thresholding at the sensors are noisy. We construct the estimators assuming that none of the diffusion parameters are nown and therefore we perform joint estimation of the source location, time of release, mass of the gas, diffusion coefficient, and sensors noise variance. We show that these estimators indeed achieve excellent performance. he rest of the paper is organized as follows. In Section II, we describe the system model and discuss the assumptions involved. he ML estimator and its calculation using Fisher s method of scoring is derived in Section III. A discussion on the performance of real-time estimation in general and the derivation of a real-time approximated ML estimator (RAML for the problem at hand are done in Section IV. he performance of the estimators is presented in Section V. Conclusions are given in Section VI. II. System Model We consider a sensor networ where L sensors are located on the real line at positions id, where i is an integer such that l i i, L l i + 1 and d is the distance between his wor was supported in part by the Office of Naval Research under Grant N and the National Science Foundation under Grant ANI consecutive sensors. Starting at time t 0, each sensor measures the atmospheric concentration of the gas in its vicinity. We assume that the sensors sample the atmosphere every seconds in a synchronized manner. A sensor outputs a 1 if the concentration exceeds a threshold γ and outputs a 0 otherwise. his thresholding, which is obviously information-lossy is required due to the need for simple cheap sensors or due to communications requirements. he output of all the sensors are available at the fusion center for processing. We assume that there is an error in the concentration measurement prior to thresholding and model this error over time and space as white Gaussian noise with zero mean and unnown variance σ 2. his noise accounts for the measurement error of the chemical sensors and the thermal noise generated in the electronic components of the sensors. We suppose that an instantaneous source of gas of mass M is released at some point x 0 on the real line at time t 0 > 0. We assume that the diffusion coefficient of the gas D is unnown. Our goal is to estimate θ x 0 t 0 M D σ, based on the binary observations available at the fusion center. We consider an idealized situation where the transport mechanism of the gas is diffusion. Subsequent to the release of the gas the mean concentration of the gas at some point x is given by m(x x 0,t t 0,D,M where m(x, t, D, M M 2 πdt e x 2 4Dt. (1 he measured mean concentration prior to thresholding R(x, t is normally distributed with mean m(x x 0,t t 0,D,M and variance σ 2. We assume that θ lies in some set Λ, such that the mean concentration m(x x 0,t t 0,D,M crosses the threshold γ in at least two sensors during the observation interval. III. he Maximum Lielihood Estimator Let y i,n {0, 1} be the output of the sensor at position id and time n. he lielihood function for a single sensor as a function of the parameters is p(y i,n ; θ (1 y i,n (1 Q (Γ i,n (θ + y i,n Q (Γ i,n (θ, (2 where Γ i,n (θ γ η i,n, (3 σ η i,n m(id x 0,n t 0,D,Mu(n t 0, (4 and u( is the step function. Since we assume the noise is white gaussian, the concentrations R(id, n and R(jd,m /04/$ IEEE 1008

2 are independent unless i j and n m. he log-lielihood function of all the sensor outputs from time up to a time 2 is then given by f 2 (θ log p(y 2 ; θ 2 n il i log p(y i,n ; θ, (5 where y 2 y,..., y 2 and y n y li,n,..., y lf,n. he ML estimator based on all the sensor outputs up to a time is then simply ˆθ ML (y 0 arg max f 0 (θ. (6 θ Λ It turns out that for certain observations vectors y 0, ˆθ ML is not unique, i.e., the log-lielihood has multiple global maxima (e.g., the all zeros sequence and the all ones sequence obviously do not give unique solution. In general, for θ Λ the probability for those sequences with non-unique solution is usually very small but not zero. Furthermore, the loglielihood has in general multiple stationary points. hese stationary points can be found from the gradient with respect to θ. he gradient of (5 is given by where and f 2 (θ α i,n (θ β i,n (θ In particular, f0 (θ 2 n (2y i,n 1α i,n(θβ i,n(θ, (7 p(y i,n ; θ il i ( 1 exp Γ i,n(θ 2, (8 2πσ 2 2 ηi,n n0 x 0 t 0 D M Γ i,n. (9 (2y i,n 1α i,n(θβ i,n(θ, (10 p(y i,n ; θ il i he solutions of (6 will be a subset of the set of the roots of (10. Unfortunately, the equation f 0 (θ 0 is a complicated non-linear equation with no closed form solution. he problem of finding zeros of a specified function is well nown in numerical analysis and among the various techniques is the Fisher s method of scoring (FS. Applying this technique, the update equation is ˆθ (n+1 ˆθ (n + µi 0(ˆθ (n 1 f 0 (ˆθ (n, (11 where I 2 (θ is the Fisher s information matrix (FIM based on the observation y 2. he FIM is given by I 2 (θ E f2 (θ f 2 (θ 2 α i,n(θ2 β i,n(θβ i,n(θ. (12 Q(Γ i,n (1 Q(Γ i,n n il i he estimator that we derived in this section estimates θ from y 0 by doing several FS iterations. Each iteration, the FIM I0 (ˆθ (n and the gradient f 0 (ˆθ (n are calculated, and the FIM is inverted. When is large, the complexity in terms of multiplications and additions is due to the calculation of FIM and the gradient, which are composed of double sums of multiplications of non-linear functions. he complexity in terms of multiplications is O(25L per FS iteration. he computational complexity of the inversion of the FIM, which is a 5 5 matrix, is negligible. If the fusion center receives new observations vector y the estimator has to perform again several iterations of (11. he computation complexity, for each iteration, thus, grows linearly with. Moreover, all the observations should be stored and the storing place also grows linearly with. he complexity, when the estimation is repeated for every new y,iso(25 2 L per FS iteration. Obviously the complexity of this estimator is large for realtime processing. his method is practical in the case that we can afford to wait for all the observations and then perform batch processing. IV. Real-ime Approximated Maximum Lielihood Estimation We first loo at the limit of the estimation performance when estimating θ from the observations y and ˆθ(y 0, where r 1 is the number of time samples between updates. he case r 1 corresponds to update every sample. he estimator uses only the previous estimate and the rl new observations, the observations y 0 are not directly available. Since y and y 0 are independent, y and ˆθ(y 0 are also independent. he log-lielihood for this estimation problem is f (ˆθ(y 0; θ log p(y, ˆθ(y 0; θ log p(y ; θ + log p(ˆθ(y 0; θ f (θ+f(ˆθ(y 0; θ. (13 It is easy to show that the FIM for this estimation problem is (θ E f (θ f (θ f(ˆθ(y +E 0; θ f(ˆθ(y 0 ; θ. (14 he first term on the right hand side of (14 is simply I (θ. he second term is the FIM of the estimation of θ based on ˆθ(y 0 and is denoted by Î(θ. Equation (14 can then be rewritten as 1 (θ I (θ+î(θ. (15 1 Note the difference between I (θ and Î (θ. In the later both ˆθ(y 0 and y are used for estimation. 1009

3 Note that since data manipulation cannot increase information, Î(θ I 0(θ. his implies that (θ I 0 (θ. (16 he bias of the estimator is b 0(θ E ˆθ(y 0 θ. (17 he covariance matrix of the estimator ˆθ(y 0is V 0(θ E (ˆθ(y 0 θ b 0(θ(ˆθ(y 0 θ b 0(θ. (18 From the CRB for biased estimators 3, 4, we get that the CRB matrix for the estimation of θ from ˆθ(y 0 is given by Ĉ(θ D 0(θÎ(θ 1 D 0(θ, (19 where D 0(θ 1 + b 0 (θ and 1 is the identity matrix. he covariance matrix of any estimator with bias b 0(θ is greater or equal than Ĉ(θ. In particular for the trivial estimator ˆθ(ˆθ(y 0 ˆθ(y 0, with the variance and bias given by (18 and (17, we get V 0(θ D 0(θÎ(θ 1 D 0(θ. (20 We assume that V 0(θ is invertible for all θ Λ. Under this assumption, we get J 0(θ D 0(θ V 0(θ 1 D 0(θ Î(θ I 0(θ. (21 From (21 and (15 we get (θ I (θ+j 0(θ. (22 From the CRB for biased estimators, we also get that C 0(θ D 0(θI 0(θ 1 D 0(θ, (23 where C 0(θ is the CRB matrix for the estimation of θ from y 0. If the estimator ˆθ(y 0 achieves this CRB, then In this case V 0(θ C 0(θ D 0(θI 0(θ 1 D 0(θ. (24 Î(θ I 0(θ (25 and (θ I (θ+i 0(θ I 0 (θ. (26 From (16, we get that (θ I 0 (θ. (27 Under this situation, the real-time approach that uses only new observations and the last estimate has the same CRB as the approach that uses all the observations. If the estimator ˆθ(y 0 does not achieve the CRB in (23, equation (22 indicates that ˆθ(y, ˆθ(y 0 may still achieve the CRB. his is true for example when ˆθ(y 0 is a sufficient statistic for the estimation of θ from the observations y 0. In order to calculate the estimator, we need to calculate (13. he term f (θ is calculated using (5, and thus we need only to calculate the term f(ˆθ(y 0; θ. Unfortunately calculating this term is practically impossible and requires maximum lielihood estimation for all the possible sequences instead of the unique observation sequence. herefore, the exact maximum lielihood estimation cannot be done in realtime more efficiently than the batch processing. o solve this problem, we continue by approximating p(ˆθ(y 0; θ as a Gaussian pdf with mean θ+b 0(θ and covariance V 0(θ. When more and more independent observations are used this assumption becomes justified. his means that y 0 should contain many observations after the release of the gas. In real-time processing, this assumption is obviously not valid in the first few iterations, where is small. A possible solution is to perform one batch processing in the beginning with such that there are enough observations after the time of release and then continue updating with the real-time estimator. It will be argued later, that this first bloc processing should be done due to the inability to estimate the parameters for a certain amount of time after the time of release. herefore, we approximate f(ˆθ(y 0; θ by 2 f(ˆθ(y 0; θ 1 2 (ˆθ(y 0 θ b 0(θ V 0(θ 1 (ˆθ(y 0 θ b 0(θ 1 2 log(det V 0(θ (28 It is straightforward to show that the FIM for the estimation of θ from ˆθ(y 0is Î(θ D 0(θ V 0(θ 1 D 0(θ. (29 herefore, with the Gaussian approximation the inequality sign of equation (22 can be replaced by an equality sign: (θ I (θ+d 0(θ V 0(θ 1 D 0(θ. (30 Using (21, (30 can be rewritten as (θ I (θ+j 0(θ. (31 With the Gaussian approximation, J 0(θ is the maximum available information from the estimate ˆθ(y 0. If J 0(θ is less than I 0(θ, the CRB of estimating θ from the observations y 0 can never be achieved. he total performance of the real-time estimator depends only on the bias and covariance of the previous estimation and the FIM of the new observations. If the estimation in any step is bad in the sense that the covariance is very large in comparison to the CRB, the whole estimation thereafter will be bad in that sense. herefore, the estimator must try to achieve the best possible performance in each step. 2 A constant term that does not depend on θ is omitted. 1010

4 We now mae some assumptions regarding b 0(θ and V 0(θ. We assume that b 0(θ 0andV 0(θ I 0(θ 1, that is the covariance of the estimator is equal to the CRB for unbiased estimators. he assumption of zero-mean is required due to the fact that the bias term cannot be calculated. he assumption that the covariance achieves the CRB for unbiased estimator for every is also required due to the fact that the covariance cannot be calculated during the estimation, since θ is obviously unnown. Based on these assumptions we now construct the approximated ML estimator. he approximated log-lielihood function is given by f (ˆθ(y 0; θ f (θ+ f(ˆθ(y 0; θ, (32 where f (θ is given in (5 for + 1 and 2 + r and f(ˆθ(y 0; θ is given in (28. he approximated gradient is given by f (ˆθ(y 0; θ f (θ + f 0 (ˆθ(y 0; θ, (33 where the term f (θ is given in (7 for + 1 and 2 + r. It is straightforward to show that f 0 (ˆθ(y 0; θ I 0(θ(ˆθ(y 0 θ. (34 We equal (33 to zero in order to find the stationary points, f (θ + I 0(θ(ˆθ(y 0 θ 0. (35 Equation (35 does not have a closed form solution and can only be solved numerically. he exact FIM required by (35 is not available since θ is not nown. hus the quantity I 0(ˆθ(y 0 is used instead. We then get the following set of update equations G (ˆθ, ˆθ (n ˆθ (n+1 ˆθ (n + (ˆθ (n f Î r+1 (ˆθ (n ˆθ (0 ˆθ ( ˆθ, (ˆθ (n I ˆθ (n+1 1 G + (ˆθ (ˆθ (ˆθ (n (n ˆθ, (ˆθ, ˆθ (n +Î r+1(ˆθ, (36 where ˆθ (n+1 (y, ˆθ, which is the (n + 1th iteration of the FS algorithm for the calculation of the estimator of θ from the observation y and the previous estimate ˆθ. ˆθ ( he previous estimate ˆθ is the value to which the previous set of iterations of the FS algorithm converge. In practice, a few iterations are enough, as long as the last iteration gets close enough to the desired value (i.e., the global maximum. he proposed estimator has a complexity in terms of multiplications of O(25rL for every FS iteration. his complexity does not depend on the time index. here is a need of constant storage place of size rl to save the new observations. he overall complexity of this method up to a time is O(25mL where m is the number of FS iterations done on each update. If r is small, the change in the estimate every update will not be large since the information contained in the new observations is relatively small. herefore, m can be chosen smaller than it is in the batch algorithm. V. Numerical Results A. ML Estimator We start with the ML estimator for the case that the distance between the sensors d is 10 m, the sampling time is 10 s, the total number of sensors L 21 and the threshold γ is 10 2 g/cm. he true values of the parameters are x 0 30 cm, t s, D 10cm 2 /s, M 1gandσ he initial values of the FS iterations are ˆx (0 0 2m,ˆt (0 0 92s, ˆD (0 15cm 2 /s, ˆM (0 5 g and ˆσ ( he estimator estimates θ from the observations y 0 where Although the initial values are not in the neighborhood of the true values the estimator converges after 570 iterations to the values ˆx cm, ˆt s, ˆD cm 2 /s, ˆM g and ˆσ In order to quantify the performance of the estimator for a specific set of parameters, we define the normalized errors e x0 ˆx(n (n 0 x 0 ˆt 0 t d, e t0 0, e D ˆD (n D D, e M ˆM (n M M and e σ ˆσ(n σ σ. We denote the vector of the normalized errors as e. he normalized error measures the resolution of the estimate. For x 0 and t 0, the resolution is relative to the spatial and temporal sampling interval. For D, M and σ, which are always positive, the errors are simply normalized by the respective true values. he overall performance of the estimator for a specific θ is given by Ee 2 x0, Ee 2 t0, Ee 2 D, Ee2 M and Ee2 σ. We are also interested in the quantities Ee x0, Ee t0, Ee D, Ee M and Ee σ. We next proceed with the approximated calculation of these quantities by averaging over different noise realizations. We would lie to compare them to the CRB for biased estimators. Due to the fact that the term D(θ does not have a close form expression and that approximating it numerically is difficult. We simply compare the performance to the CRB for unbiased estimators. Asymptotically as the spatial and temporal sampling become denser and denser and the observation interval increases, it is reasonable to assume that the bias will become smaller and smaller. If the CRB for unbiased estimators is achievable then at least we can be assured that the performance is very good although may not be the best. he results are summarized in able I. he performance of the estimation of all the parameters in terms of mean squared error outperforms the CRB for unbiased estimators. he CRBs shown in the table are actually normalized CRBs. hey are the lower bounds on the normalized mean squared error of any unbiased estimator. he estimator achieves this by trading variance for bias. It is clear from the results that the estimator is biased. We emphasize that although the comparison is not totally fair, there is no 1011

5 x 0 t 0 D M σ rue value Ee 2 db EedB CRBdB ABLE I numerical results for the ML estimator using FS method RAML ML x 0 t 0 D M σ rue value Ee 2 db EedB ABLE II numerical results for the real-time approximated ML estimator argument that the performance of the estimator in terms of localizing the source is excellent. he source can be localized, in this particular case, with a resolution of less than 1cm. hus the estimator achieves super resolution with respect to the spatial separation between the sensors. he estimation of D, M and σ is also very good. Repeating the same process for other parameter values shows similar results. B. RAML Estimator We proceed to study the performance of the RAML estimator for the the same system, same value of parameters, and same initial values. From the calculation of the CRB, it is apparent that no reliable estimation can be done from the observations y 0 for < In this case, the mean squared error (MSE in the estimation of all the parameters is large and the algorithm becomes unstable. herefore we start with initial bloc processing on the observations y 0 where his initial bloc processing should have enough iterations to converge to ˆθ ML (y 0. If this is not the case we now that we will lose information and the performance will be degraded on the average. We use 2000 FS iterations for this initial bloc processing. he complexity of the initial bloc processing is constant and does not increase with the number of observation. We then continue with the real-time update using r 10 and 30 iterations of (36 for each bloc. At the time where 20000, the estimator converges to the values ˆx cm, ˆt s, ˆD cm 2 /s, ˆM g and ˆσ he overall performance of the estimator for a specific θ is given by the mean squared error terms and the bias terms. We next proceed with the approximated calculation of these quantities by averaging over different noise realizations. he results after the 20000th iteration are summarized in able II. We obtain similar results for different values of θ. Comparing ables (I and (II, we see that the MSE is close to the MSE of the ML estimator. here is about 1dB loss for the estimation of x 0. For the other parameter the degra- 2 Ee x x 10 4 Fig. 1. he normalized error e x0 as the number of observations, dation is not more than 2.1dB. he bias terms, however, increased significantly (except from the case of t 0. his can be explained by the zero-mean approximation taen in the development of the RAML algorithm. Despite of this drawbac, the proposed real-time estimator has excellent performance in the MSE sense. It outperforms the unbiased CRB for all the parameters except x 0 and approaches the performance of the ML estimator. he large reduction in complexity may worth the loss in the performance. o demonstrate the evolution of the RAML algorithm, Figs. 1 shows the plot of Ee 2 x 0 as the number of observations,, increases. VI. Conclusions We have derived two estimators for the diffusive source localization problem: the ML and RAML algorithms which are both numerically obtained by the FS method. Both estimators exhibit excellent performance in the estimation of the source location as well as other parameters using only binary observations. Both estimators are biased and outperform the CRB for unbiased estimators by trading variance with bias. he RAML achieves performance close to that of the ML as the number of samples,, increases, with complexity of O( instead of O( 2. his reduction in complexity maes the RAML algorithm practicaor real-time signal processing. References 1 R. Ghez, A Primer of Diffusion Problems. Wiley, S. Vijayaumaran, Y. Levinboo, and. F. Wong, On diffusive source localization using dumb sensors, in Proceedings of the IEEE International Symposium on Information heory (ISI 2004, (Chicago, IL, Jun H. V. Poor, An Introduction to Signal Detection and Estimation. New Yor: Springer-Verlag, S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation heory. NJ: Prentice Hall,