Estimating the Number of Persons in an Unknown Indoor Environment by Applying Wireless Acoustic Sensors and Blind Signal Separation

Transcription

1 1 Estimating the Number of Persons in an Unknown Indoor Environment by Applying Wireless Acoustic Sensors and Blind Signal Separation Maurizio Bocca*, Cristian Galperti**, Reino Virrankoski* and Heikki N. Koivo* *Control Engineering Laboratory Helsinki University of Technology (TKK) {maurizio.bocca, reino.virrankoski, **Electronics and Information Department Politecnico di Milano Abstract Determining the number of persons and their locations in an unknown indoor environment is one of the first tasks that must be done in military, rescue or intelligence operations. If there is no functioning security system in the building, one must use a monitoring system, which is rapidly deployable but simultaneously as invisible as possible. Wireless sensor network is an attractive candidate to fill such requirements. Many sensor types are suitable for indoor situation modeling, but there are also several technical challenges rising from the limited energy and communication resources of the sensor nodes. This paper focuses on estimating the number of persons in an unknown indoor environment by using low-power acoustic sensors. The number of persons is estimated by applying blind signal separation to the acoustic signal collected by the sensor nodes. I. INTRODUCTION During recent years, rising attention has been paid to such systems that are capable to monitor situation at indoors in military, rescue, intelligence or police operations [1]. Compared to the battlefield, building interior is more complicated because many monitoring systems typically used in the battlefield environment are not feasible indoors. Furthermore, instead of looking to the grouping of forces or other macro-scale actions happening outdoors, one must figure out the locations and the actions of individual persons. That sets high accuracy requirements to the indoor situation modeling. Figuring out what is happening in the building is not an issue, if the building is equipped with a solid security system. However, we cannot assume that such a system always exists. Even in the case that the building has a security system, it can be disabled by enemy forces in a military situation, or by fire or by earthquake in a catastrophe situation. Thus, a monitoring system, which consists of such components that are rapidly deployable and attract as little attention as possible, is needed. Such requirements make wireless sensor networks an attractive choice. However, problems rising from the distributed nature of wireless sensor networks and problems rising from the limited energy resources of the sensor nodes must be solved to make them feasible for indoor situation modeling. In this paper we present an application that estimates the number of people by using the acoustic signal collected by the wireless sensor nodes equipped with a microphone. Collected signal is a mixture of several voices talking simultaneously. A whitening phase of the Independent Component Analysis is applied to determine the number of voices in the mixed signal. In addition to estimating the number of speakers, the result can be further applied to reconstruct the independent components, which are the voices of the individual persons, from the mixed signal. On the hardware side, the application requires a sample rate that is high enough to produce acoustic samples with feasible quality. Accurate time synchronization between the nodes is also required to be able to correctly fuse together the data transmitted by different nodes before applying the whitening. The paper is organized as follows. In the next section we highlight the related work. Section III describes the way how the number of source signals is computed. Section IV provides the details of the system applied in the experiments. Proposed computation method is evaluated through simulations and experiments in Section V. Section VI states our conclusions and plans for future work.

2 2 II. RELATED WORK A. Independent Component Analysis Independent Component Analysis (ICA) targets to find a linear representation of the nongaussian data so that the components are statistically independent. Assume that we have n sources transmitting simultaneously and h observing units (n h). Each observation we get is a mixture of the transmitted signals and it can be written as x i = a 11 s 1 + a 12 s a 1n s n, (1) where x i is the observation of the i:th observing unit and the source signals are noted by s i. With all observations, we get an equation group x = As, (2) where x is the vector of observations, A is the mixing matrix and s is the vector of transmitted signals. If we have a set of observations over the discrete time t = 1,...,k, equation (2) can have the same form, but so that x and s are h k matrices that include all observations. The statistical model (2) is called Independent Component Analysis or ICA-model. The independent components s i are latent variables that cannot be directly observed. After estimating the mixing matrix A, the independent components can simply be reconstructed by solving s from (2): s = A 1 x. (3) The starting point of the ICA is the assumption that the source signals s i (t) and s j (t) are statistically independent at each time moment t. However, it does not matter if the signals have overlapping spectra. It is also assumed that A is a square matrix, but that assumption is made for simplicity and it can sometimes be relaxed [2], [3]. In the Blind Signal Separation (BSS) scenario, we want to find out the original signal components but we can only observe the mixtures and we have a very little, if any, prior knowledge about the number of source signals and about the structure of the mixing matrix. ICA has become one of the most popular methods to solve the BSS problem. There exists a wide range of applications of ICA, such as biosignal processing [4], [5], finding hidden factors from financial data [6], [7], reducing the noise in the natural images [8] and signal processing in telecommunications [9]. The area of telecommunications is related to our presentation, but instead of cellular network, we are operating in the context of more resource-limited wireless sensor network. B. Sensor Network Applications In the context of wireless sensor networks, acoustic sensing is used in a wide range of applications from monitoring and detection to localization. In many applications, the duty of the sensor nodes is just to detect the existence of the acoustic pulse without analyzing the details of the detected signal. More advanced acoustic sensing is applied in the environmental monitoring and in the context recognition. One of the research projects in the Center for Embedded Networked Sensing at UCLA [10] is developing an ability to recognize and localize individual species from their vocalization. The design, analysis and testing of acoustic arrays for localizing bird vocalizations of different species is discussed in the recently reported results [11]. Kwan et al. has applied Hidden Markov Models (HMM) and Gaussian Mixture Models (GMM) to an acoustic signal to monitor and classify birds near the airport and in some critical locations in the airspace to reduce the risk of birdstrikes to the airplanes [12]. In [13], a hybrid sensor network for Cane-Toad monitoring is presented. Cane-Toads are detected based on their vocalization. In their recent article, Eronen et al. [14] investigate the feasibility of an audio-based context recognition system. They propose a system framework, in which 27 different types of urban environments are defined based on their acoustic properties. Then the defined environments are further classified to six different higher level classes. The algorithm is able to detect the surrounding environment and the shift from one type of environment to another. Environment detection is made by using the sensed acoustic signals and HMMs, which are trained by using the acoustic data collected from each type of environment a priori. ICA is also applied to extract a set of statistically independent vectors from the training data. The authors argue that such an algorithm will enable portable devices to adapt automatically to the changing surroundings. The way how our work differs from the previous ones reported in the context of sensor networks is that instead of detecting only the existence of the acoustic signal or figuring out only the higher level features of a mixed signal, we are tracing down the individual sources, i.e. the independent components, of the observed mixed signal. This presentation discusses just about estimating the number of persons by using the mixed acoustic signal, but the reconstruction of the individual voices

3 3 will be the final target and the natural continuation of this work. III. COMPUTING THE NUMBER OF SOURCE SIGNALS The process of standard ICA can be divided to the preprocessing phase and the separation phase. During the preprocessing, the measured data is first centered by subtracting its mean from it. Then a linear transformation is applied to the observation matrix X so that we obtain a new observation matrix X, which is white. In other words, the components of X are uncorrelated and their variances are equal to unity. Measurement vectors x i can be whitened by applying transformation v i = V x i. (4) In (4), the matrix V is called the whitening matrix. One of the most straightforward ways to compute the whitening matrix is by applying the Principal Component Analysis. Once we know C x, the covariance matrix of the measured data, the whitening matrix is given by V =Λ 1/2 E T, (5) where V is k x h matrix and Λ is a diagonal matrix having the eigenvalues of the covariance matrix C x in its diagonal in a decreasing order such that Λ = diag[λ 1Cx...λ kcx ]. Associated principal eigenvectors c pi are collected to matrix E =[c p1...c pk ]. Assume that we have h measured mixed signals and each of them has k measurements. Thus, we get a measurement matrix x 11 x x 1k X(k) = (6) x h1 x h2... x hk By subtracting the mean of each measurement vector from it, we get a set of measurement vectors x i (k) = (x i (k) x i (k)) and a respective measurement matrix X(k) =[ x 1 (k)... x h (k)] T. By definition, the covariance matrix of X(k) is Cov{ X(k)} = E{ X(k) X(k) T } = C x =[c ij ] = [ E{( x i x i )( x j x j ) T } ] ij, (7) where i, j =1,...,h. Since the mean of each observation vector x i is set to zero, the values in (7) become E{( x i x i )( x j x j )} = E{ x i x j } = k n=1 1 k x in x jn = 1 k ( x i1 x j x ik x jk ). (8) Thus, the covariance matrix (7) of X(k) becomes E{ x 1 (k) x 1 (k) T }... E{ x 1 (k) x h (k) T } C x =..... E{ x h (k) x 1 (k) T }... E{ x h (k) x h (k) T } x 2 = x2 1k... x 11 x h x 1k x hk. k.... x h1 x x hk x 1k... x 2 h x2 hk = 1 k X(k) X(k) T. (9) If the number of source signals is q and q h, the q largest eigenvalues of the covariance matrix (9) are some combination of the source signal powers added to the noise power, and the remaining h q eigenvalues correspond to only noise. If the signal-to-noise ratio is large enough, there is a remarkable difference of magnitude between the eigenvalues which are representing the source signals and the eigenvalues which are representing only noise [15]. Thus, in the whitening step (5) one can estimate the number of source signals q based on the difference of magnitude in the eigenvalues. This information can be used to compress the data such that we only choose the q largest eigenvalues to be used in (5) and set the remaining h q eigenvalues, that are representing only noise, to zero. In the case of mixed speech signals, the estimated value of q gives us the estimate of the number of persons in the monitored space. IV. APPLIED SYSTEM ARCHITECTURE A. Hardware The experiments are performed by using Mica2 motes equipped with a Crossbow MTS300CA sensor boards [17]. The mote with the sensor board is illustrated in the Figure 1. MTS300CA includes a low power microphone NS LMC567 [18] that can be used for general acoustic recording. B. Operation The TinyOS components for Mica2 can normally sample at a frequency up to 200 Hz. By introducing the HighFrequencySampling component [19], the sampling frequency can reach 6.67 khz, after turning off the wireless radio of the Mica2 while sampling. Each node starts the acoustic sampling in response to an external message broadcasted by the base station. The message specifies the number k and the interval m (in microseconds) of

4 λ 1 Eigenvalues of the Covariance Matrix C x λ,,λ 1 4 λ 2 λ 3 λ Length of the Considered Time Interval (s) Fig. 1. Mica2 with MTS300CA Sensor Board Fig. 2. The behavior of the covariance matrix eigenvalues when there were four source signals and eight observed mixed signals. Sampling frequency was kept on 8 khz and the total length of the sampling period is 5.5 seconds. the samples. By doing this we target to achieve the best possible accuracy in time synchronization between the nodes. During the sampling, the samples are logged over the EEPROM memory of the Mica2. After the sampling phase, the data is collected wirelessly such that the base station starts downloading data from the addressed node by transmitting another message. To avoid the lost of packages during the downloading of the data, we enabled the acknowledgement (ACK) control system. V. SIMULATIONS AND EXPERIMENTS A. Computing the Number of Source Signals In the first experiment, we used four recordings of simultaneously played different acoustic sources recorded at a frequency of 8 khz with 8 bits per sample. There were two speech signals, a rock-band playing music and a police car siren. Then we created four additional acoustic signals by mixing the previous ones and noise and ensured the statistical independence by computing that the measurement matrix had a full rank rank(x) = 8. Next we processed the measurement data and computed the covariance matrix C x according to (6)-(9). The eigenvalues of C x became λ = (10) In (10) we can clearly see the expected difference of magnitude between fourth and fifth eigenvalue indicating that there exits four different acoustic source signals in the observed mixtures. Figures 2 and 3 show the behavior of the covanriance matrix eigenvalues during the sampling period having a sampling frequency of 8 khz and a total length of 5.5 seconds. Figure 3 indicates that when such a relatively high sampling frequency is applied, the difference of magnitude in the eigenvalues shows up during the first second of the sampling. In our second experiment, we were using three Mica2 motes sampling in 3 khz frequency and two human voices talking simultaneously. The mixed signals were recorded and buffered by the motes and after sampling transmitted wirelessly to the base station. The resulting covariance matrix eigenvalues were λ = (11) The magnitude of difference between the second ( ) and third ( ) eigenvalue is still detectable even though it is not as big as in the first experiment. That result was to be expected, since in the second experiment only 3 khz sampling rate and MTS300CA Sensor Boards were used compared to the 8 khz sample rate and good quality microphones used in the first experiment. However, the result shows that it is possible to achieve a signal quality that is good enough to figure out the number of source signals and to continue with the signal separation by using resource-constrained wireless sensor nodes. B. The Effect of Time Synchronization We investigated also the effect of the time synchronization accuracy by using the three signals recorded in the second experiment. The time synchronization error between the signals was increased from zero to

5 5 Eigenvalues of the Covariance Matrix C x λ 1,,λ Length of the Considered Time Interval (s) Fig. 3. The behavior of the covariance matrix eigenvalues during the first two seconds of the sampling period. The setup is the same than the one illustrated in Figure 2. λ 1 λ 2 λ 3 λ 4 Normalized signal amplitude [-1,,1] Number of samples Fig. 5. A periodic noise component caused by the EEPROM writing component looks the same as the one in Figure 5. Eigenvalues λ 1,,λ Error in time synchronization (ms) Fig. 4. Time synchronization error effect to the covariance matrix eigenvalues. Three Mica 2 motes are applied to collect the acoustic data by using 3 khz sampling frequency. Maximum time synchronization error between the sampling motes is increased from 0 to 15 milliseconds. 15 milliseconds. The behavior of the covariance matrix eigenvalues is illustrated in Figure 4. There is no detectable difference of magnitude in the covariance matrix eigenvalues after the time synchronization error increases to three milliseconds. C. Periodic Noise Caused by the EEPROM Writing By analyzing the recordings we were able to detect the presence of a periodic noise component illustrated in Figure 5. We identified that the component is caused by writing the already collected samples to the EEPROM memory of the Mica2. Since the writing to EEPROM happens periodically, it temporarily requires more power resources and that periodic writing causes the periodic noise component to the microphone. We also strongly believe that the periodic noise component illustrated in the upper part of Figure 7 at page 5 on [13] is caused by the same reason since the authors have been using the same hardware and the noise VI. CONCLUSIONS AND FUTURE WORK In this paper, we have demonstrated the feasibility of ICA to the processing of acoustic signals in the context of wireless sensor networks. One possible application to estimate the number of persons by using the whitening phase of ICA is demonstrated by using simulations and a simple experimental setup that allows us to listen over a short period of time, transmit the acoustic signals to the base station and then perform the offline analysis. We found out that one can extract useful results by using a 3 khz sampling frequency that can be used with the wireless sensor nodes. However, once the sampling frequency is lowered from the values typically used with the wired acoustic systems, the system becomes more dependent about the accurate time synchronization. Based on our experiments, the results collected by using 3 khz sample rate become useless if the error in time synchronization exceeds three milliseconds. We are working on adding the Timing-sync Protocol for Sensor Networks (TPSN) [20] to our application. By applying the TPSN, the nodes first synchronize their clocks to the one of the base station. Once the synchronization phase is completed, the nodes start sampling simultaneously and use their clocks to time stamp the samples. To be able to listen over longer periods of time, we suggest such a clustered network architecture, where the nodes in different clusters alternate periodically between three modes: 1) sampling, 2) transmitting the samples from the node buffer to the base station and 3) repeating the time synchronization. That sort of setup allows us to keep on listening as long as there is sufficient amount of energy resources left in the sensor nodes. Developing and demonstrating such a system will be a continuation of this work. Moreover, since the resources of Mica2 are

6 6 pretty limited, using a more resource-rich node such as Telos [16] would enable more robust performance. On the algorithmic side there are two interesting paths for further research. First one is the reconstruction of the individual source signals by using ICA. Right now we are just presenting a way to estimate the number of sources, but reconstructing the individual source signals would enable us to match the speech samples against the samples stored in the police and intelligence databases. By doing so, one can define if some person whos speech sample is already stored in the database is present among the speaking persons. The second area for further research is the underconstrained case. The assumption that we must have at least as many observed mixed signals as we have sources is one of the basic assumptions in the signal separation performed by using ICA. However, the methods to separate the signals in the underconstrained case, i.e. in such a case in which there are more source signals than observed mixed signals, are under active research. Finding such a method that handles the underconstrained case and that is feasible to the sensor networks would considerably relax the overall system requirements. REFERENCES [1] Statement by Dr. Tony Tether, Defence Advanced Research Projects Agency, Submitted to the Subcommittee On Emerging Threats And Capabilities, Committee on Armed Services, United States Senate, March 9, 2005 [2] A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5): , 2000 [3] A. Hyvarinen, J. Karhunen and E. Oja, Independent Component Analysis, Jhon Wiley & Sons, New York, 2001 [4] R. Vigario, Extraction of Ocular Artifacts from EEG Using Independent Component Analysis, Electroenceph. Clin. Neurophysiol., 103(3): [5] R. Vigario, V. Jousmaki, M. Hamalainen, R. Hari and E. Oja, Independent Component Analysis for Identification of Artifacts in Magnetoencephalographic Recordings, Advances in Neural Information Processing Systems, vol. 10, pp , MIT Press, 1998 [6] A. D. Back and A. S. Weigend, A First Application of Independent Component Analysis to Extracting Structure from Stock Returns, Int. J. On Neural Systems, 8(4): , 1997 [7] K. Kiviluoto and E. Oja, Independent Component Analysis for Parallel Financial Time Series, In the Proceedings of International Conference on Neural Information Processing (ICONIP98), vol. 2, pp , Tokyo, Japan, 1998 [8] A. Hyvarinen, Sprase Code Shrinkage: Denosing of Nongaussian Data by Maximum Likelihood Estimation for Independent Component Analysis, Neural Computation, 11(7): , 1999 [9] T. Ristaniemi and J. Joutsensalo, On the Performance of Blind Source Separation in CDMA Downlink, In the Proceedings of International Workshop on Independent Component Analysis and Blind Signal Separation (ICA99), pp , Aussosis, France, 1999 [10] CENS Website: [11] C-E. Chen, H. M. Ali and H. Wang Design and Testing of Robust Acoustic Arrays for Localization and Enhancement of Several Bird Sources, In the Proceedings of Information Processing in Sensor Networks (IPSN06), April 19-21, Nashville, Tennessee, USA, 2006 [12] C. Kwan, K. Ho, G. Mei, Y. Li, Z. Ren, R. Xu, G. Zhao, M. Stevenson, V. Stanford and C. Rochet, An Automated Acoustic System to Monitor and Classify Birds, Presentations of the 5th annual meeting of the Bird Strike Committee-USA/Canada, Toronto, Ontario, Canada, August 18-21, 2003 [13] W. Hu, V. N. Tran, N. Bulusu, C. T. Chou, S. K. Jha and A. Taylor, The Design and Evaluation of a Hybrid Sensor Network for Cane-Toad Monitoring, In the Proceedings of the Fourth International Symposium on Information Processing in Sensor Networks, IPSN 2005, UCLA, Los Angeles, California, USA, April 25-27, 2005 [14] A. J. Eronen, V. T. Peltonen, J. T. Tuomi, A. P. Klapuri, S. Fagerlund, T. Sorsa, G. Lorho and J. Huopaniemi, Audio-Based Context Recognition, IEEE Transactions on Audio, Speech and Language Processing, Vol. 14, No. 1, January 2006 [15] F. M. Ham, N. A. Faour and J. C. Wheeler, Infrasound Signal Separation Using Independent Component Analysis, SPIE, Vol. 4055, Orlando, Florida, USA, April 2000 [16] J. Polastre, R. Szewczyk and D. Culler, Telos: Enabling Ultra- Low Power Wireless Research, In the Proceedings of The Fourth International Conference on Information Processing in Sensor Networks: Special track on Platform Tools and Design Methods for Network Embedded Sensors (IPSN/SPOTS), Los Angeles, California, USA, April 25-27, 2005 [17] Crossbow Website: [18] National Semiconductor LMC567 Low Power Tone Detector: [19] S. Kim, D. Culler and J. Demmel, Structural Health Monitoring Using Wireless Sensor Networks, Berkeley Deeply Embedded Network System Course Report, 2004 [20] S. Ganeriwal, R. Kumar and M. B. Srivastava, Timing-sync Protocol for Sensor Networks, SenSys 2003, Los Angeles, California, USA, November 5-7, 2003