ICSV14. Cairns Australia 9-12 July, 2007 DIRECT ALGEBRAIC METHOD FOR SOUND SOURCE LOCALIZATION WITH FINEST RESOLUTION BOTH IN TIME AND FREQUENCY

Transcription

1 ICSV14 Cairns Australia 9-12 July, 2007 DIRECT ALGEBRAIC METHOD FOR SOUND SOURCE LOCALIZATION WITH FINEST RESOLUTION BOTH IN TIME AND FREQUENCY Shigeru ANDO, Nobutaka ONO, and Takaaki NARA Department of Information Physis and Computing, University of Tokyo Hongo, Bunkyo-ku, Tokyo, , Japan Abstrat We deal with the problem of diretion and distane estimates of sound soures in 3-D spae and arrive at a new exat and diret algorithm from finite observations both in spae and time. We first derive a partial differential equation (PDE) whih we all the sound soure onstraint (SSC). We show that the general solution of the SSC-PDE is a diverging spherial wave from a point soure with arbitrary temporal waveform. The SSC enables the observer to determine the soure loation (distane R and diretion n) from loal measurements of the wavefield. For the measurements of wavefield, we onsider weighted temporal integrals of arrayed mirophone outputs in a finite duration. We obtain exat formulae for loalizing a single soure from single weight measurements and multiple soures from the ombination of differently weighted measurements. We examine the performane by simulating non-stationary omplex multi-soure environments. 1. INTRODUCTION Diretional sensing of sounds enables the loalization of their soures in spae. Sound soure loalization is one of the most important funtions of auditory systems. It an aid in the separation of signals from multiple soures and in their identifiation. Appliations inlude the loalization and traking of speakers in onferene rooms, improved hearing aids having diretional sensitivity, and the realization of ears of mobile robots to endow them with loalizing, separating, and ommuniating apability[1] even for moving soures and mirophones[2]. Most loalization methods depend on two types of physial variables derived from sensor signals: time delay of arrival (TDOA) and diretion of arrival (DOA). DOA an be estimated by exploiting the phase differene at sensors and is appliable to narrow band soures. Typial algorithms for multiple soures inlude ESPRIT[4] and MUSIC[3] based on the subspae deomposition of ovariane matrix of reeived signals. TDOA is appliable to broadband soure and relies mostly on on the aurate measurement of time delays between pairs of mirophones based on the ross orrelation. Consistent triangulation of all delays produe the soure distribution[5, 6].

2 S. Ando, N. Ono, and Y. Fujita However the problems of existing methods are in several aspets. Firstly they are based on statistial quantities (ovariane of arrayed sensors or orrelation funtions of reeived signals), hene long observation duration is required for their stability. Otherwise the result beomes noisy with numerous spurious soures. It also worsens the mixture ondition of multiple soures both in time and frequeny, and makes inapable to use inherent granular struture in these domains of most sound soures like human voie and environmental noise. Another problem is that no diret algorithm is available for onventional types of mirophone array. Iterations for global optimization or an entire sweep of sound field are neessary. This requires the use of massive and ostly omputing power, whih is not suitable for implementation in most pratial systems and appliations. The purpose of this study is therefore to develop an algorithm with following properties and show its signifiane in omplex, noisy, and reverberant environments being ommon in realworldappliations. The first issue is on the usage of data. The algorithm should not rely on statistial quantities among them but proess the waveform data diretly. The seond issue is on the exatness of algorithm for finite observation. By it, the use of any short observation duration is possible without onerning errors due to theoretial approximation. The third issue is on the diretness of the algorithm. An expliit, losed-form algorithm provides the solution diretly and effiiently. This exludes massive and ostly hardwares, and makes muh easier its implementation in tiny sensor nodes. In the next setion, we begin with the fundamental equation in the form of PDE, and integrate it to obtain an algebrai algorithm. 2. SOUND SOURCE CONSTRAINT PDE Let the loation vetors of the observation point P and an only one soure S be r =(x, y, z) and r 0 =(x 0,y 0,z 0 ), respetively. Let the sound pressure of the soure be g(t). Then, from the general spherially symmetri solution of the wave equation, the pressure on P is expressed as 1 f(r,t)= r r 0 g(t r r 0 ), (1) where is the sound veloity, and r r 0 = {(x x 0 ) 2 +(y y 0 ) 2 +(z z 0 ) 2 } 1/2. (2) Let the unit diretion vetor from P to S be n. The pressure gradient at P is expressed as 1 f(r,t)= r r 0 2 g(t r r 0 1 ) n + r r 0 ġ(t r r 0 ) n, (3) and the temporal gradient is f(r,t)= 1 r r 0 ġ(t r r 0 ). (4) Therefore, substituting Eq.(1),(4) to Eq.(3) to eliminate the soure waveform g(t), we obtain an equation of only the pressure and its gradient at the observation point P as [7, 8] 1 f(r,t)={ r r 0 f(r,t)+1 f(r,t)} n = { 1 R f(r,t)+1 f(r,t)} n, (5)

3 ICSV14, July 9-12, 2007, Cairns, Australia where R r r 0 is the distane from P to S. We hereafter all this equation the sound soure onstraint (SSC) PDE. For the SSC-PDE, we an verify the following property. Theorem 1 The general solution of the SSC is a spherial wave with arbitrary temporal waveform, whih is emitted from a soure (R, n). This theorem assures that the SSC-PDE provides both neessary and suffiient desription of the sound field that enables to determine the sound soure. An only freedom left in the SSC-PDE is the waveform. For the desired loation information, it onstrains it determinately. Therefore the next subjet is to obtain an algebrai equation that relates the measurement quantities and the soure loation. This requires anyways an integration of the SSC-PDE. 3. INTEGRAL SSC-PDE AND ITS SOLUTION We onsider the integration in time axis. The gradient in spatial axis is left as it is. We assume they are provided by one of the gradient measurement tehniques [10, 11, 12, 13, 19, 20]. 3.1 Integral form of SSC-PDE Assume that the loation of sound soure is stationary during a finite observation interval [,T/2] (T is the observation time e.g. 2ms, 2.9ms, or 32ms in the experiments of setion V). The sound waveform is arbitrary inluding its on/off. Then the SSC-PDE is satisfied anywhere in [,T/2]. In this ase, we an invoke the identity relation with an arbitrary weighting funtion w(t) as f ( 1 R f + 1 f)n = 0 t [ T 2, T 2 ] { f ( 1 R f + 1 f)n}w(t)dt w(t), (6) where the variables (r,t) of f are omitted for brevity. Atually, the weighted integral form in the seond line beome idential with the SSC-PDE in the first line when we hoose {w(t)} as a omplete set of funtion in [,T/2] and solving them simultaneously. Using this relation, we an integrate the SSC-PDE in time axis as follows. First, we introdue the omplex exponential funtions {e jωt } as the set of weighting funtions. Why we hoose e jωt is readily larified in the ourse of derivation. The measurement quantities are the weighted integrals of the sound field at and near the observation point r as g ω (r) f(r,t)e jωt dt. (7) Integration of the SSC-PDE with the weight funtion e jωt in [,T/2] yields { f(r,t) ( 1 R f(r,t)+1 f(r,t))n}e jωt dt

4 S. Ando, N. Ono, and Y. Fujita = f(r,t)e jωt dt n f(r,t)e jωt dt n R [ f(r,t)e jωt ] T/2 jωn = g ω (r) n R g ω(r) n = g ω (r) n R g ω(r) n [ f(r,t)e jωt ] T/2 = 0, jωn g ω(r) e jωt t f(r,t)dt f(r,t)e jωt dt where t / t for brevity. It should be noted that the weighted integral with the weight t e jωt beomes equal to the one with e jωt exept for the oeffiient jω by hoosing the omplex exponential funtions as the weight. This signifiantly redued the number of measurement quantities. Further more, the integral boundary term [ f(r,t)e jωt] T/2 an also be eliminated by hoosing ωt =2nπ (n =0, 1, 2, ) whih is atually the orthogonal omplete ondition of {e jωt }.Itis beause [ f(r,t)e jωt ] T/2 = f(r,t/2)e jπn f(r, )e jπn =( 1) n (f(r,t/2) f(r, )) =( 1) n[ f(r,t) ] T/2, whih shows the integral boundary term for all weight frequenies ω =2nπ/T (n =0, 1, 2, ) are equal exept for the sign s n ( 1) n. Therefore we obtain a omplex vetor equation g ω (r) ={ ( 1 R + jωs n ) gω (r)+ s n [ ] T/2 f(r,t) }n (8) with unknown variables n,r, [ f(r,t) ] T/2. We all this equation the (temporal) integral form of sound soure onstraint (issc). The issc, as well as the SSC-PDE, provides an exat relation to determine the soure loation. Furthermore, it provides an algebrai relation for a diret solution based on the weighted integral measurements in ever so a small interval [,T/2]. Indeed the weighted integral is the well-known Fourier series oeffiient, but no smooth window funtion (exept for the do-nothing window) is required to obtain them. 3.2 Linearized algorithm Sine the issc has = 4non-observable degrees of freedom (dof) whereas the number of independent equations are 6 or 4 for 3-D or 2-D gradient measurement of g ω (r), we an essentially solve for the the soure loation using only one omplex vetor equation for a single weight frequeny ω. However, Eq.(8) is nonlinear for R, [ f(r,t) ] T/2 and n beause of their multipliations involved. To obtain rapidly an initial guess, desired is the linearization of Eq.(8) based on the extension of unknown variables into n/r, n/, n [ f(r,t) ] T/2. In this ase, the number of unknowns is 3+3+3= 9for three-dimensional (3-D) gradient measurement and 2+2+2= 6for 2-D

5 ICSV14, July 9-12, 2007, Cairns, Australia gradient measurement. This means two frequenies are suffiient for both ases to loalize a single soure. Two simple ases are the use of DC frequeny and adjaent frequeny as one of the paired frequenies. For the DC frequeny ase, sine s 0 =1, it follows that g 0 (r) ={ 1 R g ω(r)+ 1 hene, eliminating [ f(r,t) ] T/2, we obtain simply as [ f(r,t) ] T/2 }n (g ω (r) s n g 0 (r)) = { 1 R (g ω(r) s n g 0 (r)) + jω g ω(r)}n. (9) For the use of adjaent frequenies ω 1,ω 2, it follows that (s 2 g ω1 s 1 g ω2 )={ s 2 R g ω1 s 1 R g ω2 + jω 1s 2 g ω1 jω 2s 1 g ω2 }n, (10) where s 1,s 2 are the values of s n for ω 1,ω 2 respetively. In both equations, the integral boundary terms are eliminated and n/r, n/ an be solved diretly. Sueeding nonlinear least squares estimation from n/r, n/ (4 dof) to R, n (3 dof) redues the noise and improves the auray. 4. FINE GRAIN LOCALIZATION IN TIME AND FREQUENCY Most established methods for the multiple sound soure loalization, e.g. the eigenspae methods[3, 4], require long observation time to obtain narrow frequeny bands and well-onverged statistial quantities, whih an redue signifiantly the temporal resolution. Reently, another approah appears that relies on the time-frequeny sparseness of sound power distributions to separate their soures. For example, the power of speeh sounds is onentrated on some fragments of the timefrequeny domain beause of the harmoni struture, formants, glottal losures, et. Where the soure signals have sparseness, they are rarely overlapped thus one soure is only ative almost everywhere. This means in a small time-frequeny, i.e. a grain, single soure loalization is suffiient. Aumulation of those results yields the multiple loalization with the finest granularity both in time and spae. One important point of this approah is that the nature of sparseness an hange soure by soure. For impulsive sounds like mehanial noise, high temporal resolution (short frame length) is appropriate. Some kind of slowly hanging periodi noise are, however, isolated well by a narrow band analysis (long frame length). In the short time Fourier transform (STFT) analysis for speeh, the optimal frame length is about 1024 sample for the 16kHz sampling frequeny[21] beause of its moderately hanging nature. The PDE-based framework that provides always an exat formula irrespetive of the length of analysis frame is therefore suitable inherently for the sparseness-based multiple soure loalization. To adjust the time-frequeny resolution to the target signal, we an use the following relations ω =2π/T, Ω=2π/ t, (11) where T is the time resolution of time-varying spetra (observation interval), ω is the frequeny resolution, and Ω is the maximal analysis frequeny of sounds.

6 S. Ando, N. Ono, and Y. Fujita 1.0m 0.8m 0.4m 1.0m sensors 0.4m sensors (A) (B) Figure 1: Two experimental setups and the olor samples for display of results frequeny [Hz] frequeny [Hz] times [s] times [s] (a) 512 points STFT (T =32ms) (b) 32 points STFT (T =2ms) Figure 2: The estimated soure diretion maps in time-frequeny plane for (a) setup A with the 512 points (32ms) STFT, and for (b) setup A with the 32 points (2ms) STFT. The soure diretion in eah grain is indiated by a olor shown by Fig.2 left irle. 5. NUMERICAL EVALUATION Fig. 1 shows simulated two environments for experiments of sound soure loalization. In the setup A, four soures are loated in the same distane and the different diretions. While in the setup B, two soures are loated in the different distanes and the same diretion. In both of them, soure signals were different speeh utteranes sampled by 16kHz. Spatial gradients were obtained by the differene of observed signals at 5m-spaed points. The sound soure diretion and distane were estimated at every time-frequeny bin by our method. For eah setup, narrow band and wide band analysis were examined. Fig. 2(a) and Fig. 2(b) show the estimated soure diretions for the setup A as olor maps. They were obtained through 512 or 32 points STFT, respetively. Mainly, due to the harmoni struture in the narrow band analysis, or the temporal pith struture in the wide band analysis, the energies of the speeh utteranes are sparsely distributed and rarely overlapped. Thus, in the two figures, four olors orresponding to the four soures are learly dominant, whih indiates that the soures are suessfully loalized. Apart from differenes of time and frequeny resolutions,

7 ICSV14, July 9-12, 2007, Cairns, Australia the estimated diretions are almost onsistent in the two figures. Fig. 3(a) and Fig. 3(b) also show the estimated distanes for the setup B in the same way. It illustrates that the disrimination by the soure distane is well performed frequeny [Hz] frequeny [Hz] times [s] times [s] (a) 512 points STFT (T =32ms) (b) 32 points STFT (T =2ms) Figure 3: The estimated soure distane maps in time-frequeny plane for (a) setup B with the 512 points (32ms) STFT, and for (b) setup B with the 32 points (2ms) STFT. The soure diretion in eah grain is indiated by a olor shown by Fig.2 right bar. Atually, the optimal time and frequeny resolution for the sparse representation depends on statistial properties of target, noise, or surrounding environment like reverberation. However, in the onventional sparseness-based loalization, the high frequeny resolution is essential to estimate time-delay from phase differene, whih limits the performane of the sparseness-based loalization. While in our method, arbitrary resolution is allowable as shown in this experiments beause of the rigorous formulation of finite observation. The advantage should yield the apability of the robust loalization in the real environment. 6. SUMMARY We proposed a novel algorithm of sound soure loalization with following properties: 1) use of not statistial but wideband waveform data diretly, 2) exatness of algorithm for any finite and short observation, and 3) diretness of the algorithm without iterations or DOA sweeps. We examined the algorithm with numerially simulation and an atual data taken in a room with signifiant reverberation. Referenes [1] Y. Senjo, K. Miyamoto, T. Kurihara, and S. Ando, A novel oupling sheme of speeh separation and fae reognition: Implementation with SmartHead autonomous vision sensor with ears, Pro. Int. Workshop Networked Sensing Systems (INSS 2006), Chiago, [2] T. Horiuhi, M. Mizumahi, and S. Nakamura, Iterative estimation and ompensation of signal diretion for moving sound soure by mobile mirophone array, IEICE Trans. Fundamentals, vol.e87-a, no.11, pp , 2004.

8 S. Ando, N. Ono, and Y. Fujita [3] R. O. Shmidt, Multiple Emitter Loation and Signal Parameter Estimation, IEEE Trans. Antennas and Propagation, vol.34, no.3, [4] A. Paulraj and R. R. Kailath, ESPRIT-Estimation of Signal Parameters via Rotational Invariane Tehniques, IEEE Trans. Aoust., Speeh, Signal Proessing, vol.37, no.7, pp , [5] J. O. Smith and J. S. Abel, Closed-form least-squares soure loation estimation from range-differene measurements, IEEE Trans. Aoust., Speeh, Signal Proessing, vol.35, no.12, pp , [6] M. S. Brandstein, J. E. Adok, and H. F. Silverman, A losed-form loation estimation for use with room environment mirophone arrays, IEEE Trans. Speeh and Audio Proessing, vol.5, no.1, pp.45-50, [7] S. Ando and N. Ono, Partial differential equation (PDE)-based theory of sound soure loalization, 4th Joint Meeting ASA and ASA, Honolulu, [8] Y. Fujita, N. Ono, and S. Ando, Partial-differential-equation-based sound soure loalization: Finite Fourier integral approah and its appliation to multiple soure loalization, 4th Joint Meeting ASA and ASJ, Honolulu, [9] T. Nara and S. Ando, Projetive Method for the Inverse Soure Problem of the Poisson Equation, Inverse Problems, vol.19, no.2, pp , [10] S. Ando, H. Shinoda, K. Ogawa, and S. Mitsuyama, A Three-Dimensional Sound Loalization System Based on the Spatio-Temporal Gradient Method, Trans. So. Instrumentation and Control Engineers, vol.29, no.5, pp , (in Japanese) [11] S. Ando, An Intelligent Three-Dimensional Vision Sensor with Ears, Sensors and Materials, vol.7, no.3, pp , [12] S. Ando, An Autonomous Three-Dimensional Vision Sensor with Ears, Trans. IEICE, vol.e78-d, no.12, pp , [13] S. Ando and H. Shinoda, Ultrasoni Emission Tatile Sensing, IEEE Control Systems, vol.15, no.1, pp.61-69, [14] N. Ono and S. Ando, Sound Soure Loalization Sensor with Mimiking Barn Owls, Pro. Transduers 01, vol.2, pp , Munih, Jun [15] N. Ono, T. Hirata, and S. Ando, A Study on Sound Soure Loalization Sensor with Mimiking Ormia Ohraea s Ears, Teh. Digest 18th Sensor Symp., pp , May [16] N. Ono, Y. Zaitsu, T. Nomiyama, A. Kimahi and S. Ando, Biomimiry Sound Soure Loalization with Fishbone, Trans. IEEJ(E), vol.121-e, no.6, pp , Jun [17] M. Kurihara, N. Ono, and S. Ando, Theory and experiment of dual sound soure loalization with five proximate mirophones, Pro, SICE Annual Conferene, pp , [18] N. Ono, A. Saito, and S. Ando, Bio-mimiry sound soure loalization with gimbal diaphragm, Pro. 19th Sensor Symp., pp , Kyoto, May [19] N. Ono, A. Saito, and S. Ando, Bio-mimiry Sound Soure Loalization with Gimbal Diaphragm, Trans. IEEJ, vol.123-e, no.3, pp.92-97, [20] N. Ono, A. Saito, and S. Ando, Design and Experiments of Bio-mimiry Sound Soure Loalization Sensor with Gimbal-Supported Cirular Diaphragm, Pro. 12th Int. Conf. Solid State Sensors and Atuators (Transduers 03), pp , [21] O. Yilmaz and S. Rikard, Blind Separation of Speeh Mixtures via Time-Frequeny Masking, IEEE Trans. Signal Proessing, vol. 52, no. 7, pp , 2004.