Introduction to distributed speech enhancement algorithms for ad hoc microphone arrays and wireless acoustic sensor networks

Transcription

1 Introduction to distributed speech enhancement algorithms for ad hoc microphone arrays and wireless acoustic sensor networks Part IV: Random Microphone Deployment Sharon Gannot 1 and Alexander Bertrand 2 1 Faculty of Engineering, Bar-Ilan University, Israel 2 KU Leuven, E.E. Department ESAT-STADIUS, Belgium EUSIPCO 2013, Marrakesh, Morocco S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

2 Special Thanks This presentation is partly based on the Ph.D. dissertation by Shmulik Markovich-Golan S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

3 Synchronization Problem formulation Blind Sampling Rate Offset Estimation and Compensation [Markovich-Golan et al., 2012] Scenario Fully connected N nodes network with M n microphones at the nth node. Nominal sampling rate f s. Sampling rate f s,n = (1 + ɛ n ) f s, sampling period T s,n with Sampling rate offset ɛ n. TF-GSC [Gannot et al., 2001] with Sampling Rate Offsets RTF is constantly changing: signal distortion. ANC is constantly updating: increased noise level. Microphone signals are less coherent: degraded performance. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

4 Synchronization Solution sketch Block Diagram of Synchronized TF-GSC Input Offsets compensation TF-GSC Output Offsets estimation Synchronized TF-GSC Sampling rate estimation: based on the phase drift of the coherence between microphones in stationary noise-only segments (in coherent frequency bands). Resampling with Lagrange polynomials interpolation [Erup et al., 1993]. Other beamforming sync. methods: [Wehr et al., 2004]; [Ono et al., 2009]; [Liu, 2008]. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

5 Synchronization Offset estimation at the nth node Continuous Microphone Signals Received noise component at microphone s, 1st node: v 1,s (t). Received noise component at microphone r, nth node: v n,r (t). Noise only time-segment z 1,s (t), f s =1 z n,r (t), f s,n = t S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

6 Synchronization Offset estimation at the nth node Statistics of Noise Components v 1,s (t) and v n,r (t) Cross-covariance: R s,r (τ) = E {v 1,s (t) v n,r (t τ)}. Cross-spectrum: θ s,r (ξ) = R s,r (τ) exp ( jξτ) dτ θ s,r (ξ) 2 1 R s,r (τ) θ s,r (ξ) [rad] τ Cross-covariance ξ Cross-spectrum S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

7 Synchronization Offset estimation at the nth node Sampled Microphone Signals v 1,s [l] = v 1,s (lt s ). v n,r [l] = v n,r (lt s,n ) z 1,s (t), f s =1 z n,r (t), f s,n = t S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

8 Synchronization Offset estimation at the nth node Statistics of Microphones v 1,s (t) and v n,r (t ) Cross-covariance: R s,r (τ + ). Cross-spectrum: θ s,r (ξ) = exp (jξ ) θ s,r (ξ). Time difference at the lth sample: = lt s lt s,n lt s ɛ n (using first-order Taylor series approximation) R s,r (τ) 3 θ s,r (ξ) 0.3 R s,r (τ+δ) 2 θ Δ (ξ) s,r ( ) [rad] τ Cross-covariance ξ Cross-spectrum S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

9 Synchronization Offset estimation at the nth node Statistics of Sampled Microphones v 1,s [l] and v n,r [l] Cross-spectrum is band-limited by fs 2 Assume small offset. θs,r l ( [k] = θs,r ltsɛn 2πkfs ) K. ( Let, θ s,r [k] = θ 2πkfs ) s,r K, then: ( θs,r l [k] = exp j 2πklɛ ) n θ s,r [k] K Coherence between microphones s and r at the lth sample Define γ l s,r [k] = θ l s,r [k] θs,s [k]θ r,r [k] and γ s,r [k] = k = 0, 1,..., K 1. Then γs,r l [k] = αnγ l s,r [k] with α n = exp ( ) j 2πkɛn K. ɛ n can be extracted from α n. θ s,r [k] θs,s [k]θ r,r [k] ; for S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

10 Synchronization Offset estimation at the nth node Offset Estimation at the nth Node Given a noise-only time segment of P s samples. Partition into I frames of P samples: l = i P; i = 0, 1,..., I 1. estimate M 1 M n coherence matrix ˆΓ ip 1,n [k] between microphones of the 1st and the nth nodes P 1 P 2P 1 ( I 2) P ( I 1) P 1 l 0 1 I 1 ˆ 0 1, n Γ [ k] ˆ P 1, n Γ [ k ] ˆ I Γ ( 1) P 1, n [ k] ɛ n < ɛ max no 2π ambiguity for k k max = K 2Pɛ max. Estimate the nth node sampling rate offset: ( s, r pair: ˆɛ n,s,r = avg K k 2πPk avg ˆγ s,r ip [k]/γ (i 1)P ) s,r [k]. i }{{} Average all microphone pairs: ˆɛ n = 1 M 1M n M1 s=1 ˆα P n Mn r=1 ˆɛ n,s,r. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

11 Synchronization Offset compensation at the nth node Resampling with Lagrange Polynomials Interpolation [Pawig et al., 2010],[Erup et al., 1993] I Resample z n,r (pt s,n ) to z n,r (pt s ) Interpolate z n,r [p] by factor 4: z n,r [ p]. Denote: ṗ = 4pTs T s,n = 4p (1 + ˆɛ n ), the closest interpolated sample index from the left to time pt s. The resampled value of z n,r (pt s ) is ẑ n,r [p] = β p 1 z n,r [ṗ 1] + β p 0 z n,r [ṗ] + β p 1 z n,r [ṗ + 1] + β p 2 z n,r [ṗ + 2]. Input Lagrange Output 4 LPF interpolation znr, p z p ˆn, r S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

12 Synchronization Offset compensation at the nth node Resampling with Lagrange Polynomials Interpolation [Pawig et al., 2010],[Erup et al., 1993] II Calculate Weights η = 4p (1 + ˆɛ n ) ṗ. β p 1 = η(η 1)(η 2) 6. β p 0 = (η+1)(η 1)(η 2) 2. β p 1 = (η+1)η(η 2) 2. β p 2 = (η+1)η(η 1) , 1, S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

13 Synchronization Experimental study Experimental Study Q directional stationary interfering sources TF-GSC Algorithms W.o. offsets; Conventional TF-GSC; Synchronized TF-GSC Criteria Signal to Distortion ratio (SDR); Signal to Noise (SNR) Without offset With offset Conventional Conventional Synchronized Q SDR SNR Ex. Ex. Ex. Ex. Dist. Noise Dist. Noise Values in db, Ex. - excess values S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

14 Statistical Beamformer Problem formulation & definitions WASNs with Random Node Deployment [Markovich-Golan et al., 2011]; [Markovich-Golan et al., 2013]; general reading [Lo, 1964] Scenarios Ad hoc sensor networks. Large volume (and many microphones). High fault percentage. Arbitrary microphone deployment. Questions How many microphones are required? What is the expected performance? Is there an optimal deployment? [Kodrasi et al., 2011] S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

15 Statistical Beamformer Problem formulation & definitions Outline Array of randomly located microphones in a reverberant enclosure. Single desired speaker. Utilizing the statistical model of the ATFs, statistical models for the SIR and WNG are derived. The reliability of the SDW-MWF is computed for: Multiple coherent noise sources. Diffuse sound field. The reliability functions can be used to determine the number of microphones required to assure a desired performance level (with a controlled level of uncertainty). S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

16 Statistical Beamformer Problem formulation & definitions Notations I Signals Let s d (l, k) be a desired speaker signal located at r d. Microphone signals: z(l, k) h d (l, k) s d (l, k) + v(l, k). Microphone signals PSD: Φ zz (l, k) E{z(l, k)z H (l, k)} = σd 2 (l, k)h d(l, k)h H d (l, k) + Φ vv(l, k) Noise PSD: Φ vv (l, k) E{v(l, k)v H (l, k)}. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

17 Statistical Beamformer Problem formulation & definitions Notations II Room Constellation Room volume and surface area: V D x D y D z Reverberation time: T 60. A 2 (D x D y + D x D z + D y D z ) M microphones randomly deployed with a uniform distribution at coordinates r m [ rx m ry m rz m ] T ; m = 1,..., M. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

18 Statistical Beamformer Problem formulation & definitions Criterion SDW-MWF ( (w w argmin 1 ) ) H hd 2 σd 2 + µ ( w ) H Φvv w = w SINR and WNG are Random Variables Signal to Interference and Noise (SINR): Φ 1 vv h d h H d Φ 1 vv h d + µ σ 2 d κ σ2 d wh h d 2 w H Φ vv w = σ2 d hh d Φ 1 vv h d White noise gain (WNG): ( ξ wh h d 2 h H w 2 = d Φ 1 ) 2 vv h d h H d Φ 2 vv h d S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

19 Statistical Beamformer Statistical ATF modelling Statistical ATF Modelling ATF relating a coherent source at r d, and the mth microphone at r m h h + ĥ h the direct arrival. ĥ the reverberant component. The direct arrival and the reverberant tail assumed uncorrelated. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

20 Statistical Beamformer Statistical ATF modelling Reverberant Tail Model [Schroeder, 1987],[Kuttruff, 2000] Under the Assumptions: The signal wavelength is much smaller than the room dimensions. The microphones and sources are at least half wavelength away from the walls. The signal frequency is above the Schroeder frequency, f Schroeder 2000 (typically few hundred Hz). The Tail Statistics: with ˆα 1 ε πεa T 60 V ĥ CN (0, ˆα) 0.161V and ε AT 60, the exponential decay rate of the RIR tail. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

21 Statistical Beamformer Statistical ATF modelling The Direct Arrival (Spherical Wave Propagation) The Direct Arrival Model h ā exp ( j φ ) where: { 1 ; rd 1 ā = 4π 1 4π r d r m ; 1 4π < r d φ = 2π r d r m λ k S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

22 Statistical Beamformer Statistical ATF modelling Second-Order Statistics of single ATF Arbitrary Sensor Location within r Under the Assumptions: For r r c, where r c path is negligible [Kuttruff, 2000]. V 100πT 60 is the critical distance, the direct For r λ k multiple 2π phase cycles are repeated while sound wave propagates. r arbitrarily chosen (the results are not sensitive to the exact value). Approximations: E { h } 0 E {h} = 0. E { h 2} α = 4π r 3 6π r 1 3V ᾱ + ˆα with ᾱ = 32π 3 r. 3 The ATFs h m ; m = 1, dots, M relating the source and randomly deployed microphones are i.i.d. (for large sphere and few microphones). S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

23 Statistical Beamformer Statistical ATF modelling Covariance of ATFs Relating 2 Sources and Randomly Located Microphone ATFs covariance: E {h 1 h 2} = E { h 1 h 2} + E {ĥ1 ĥ 2 } Reverberant tail is diffused [Jacobsen and Roisin, 2000]: } ( ) E {ĥ1 ĥ2 2π r1 r 2 = ˆα sinc Assuming r 1 r 2 λ k : } E {ĥ1 ĥ2 0, since the sinc is decaying. E { h1 h } 2 0, since multiple 2π phase cycles are repeated while sound wave propagates. λ k S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

24 Statistical Beamformer Statistical ATF modelling Reliability Measures SIR Reliability The reliability of an SIR level of κ 0 is defined as the probability that the output SIR will exceed κ 0 : R κ (κ 0 ) Pr (κ κ 0 ). White Noise Gain (WNG) Reliability The reliability of a WNG level of ξ 0 is defined as the probability that the WNG will exceed ξ 0 : R ξ (ξ 0 ) Pr (ξ ξ 0 ). S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

25 Statistical Beamformer Statistical ATF modelling P Directional Noise Sources For high INR and P M ( 2 σ 2 ) u R κ,c (κ 0 ) =1 F η,c α σd 2 κ 0 ( ) 2 R ξ,c (ξ 0 ) =1 F η,c α ξ 0. σu 2 - sensor noise variance and σd 2 - desired source varaince. η c χ 2 (2 (M P)) Chi-square RV with 2 (M P) degrees of freedom. F η,c (η 0 ) Pr (η c η 0 ) = γ f (M P, η 0 2 ) Γ f (M P) is the respective CDF. Γ f is the Gamma function. γ f is the lower incomplete Gamma function. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

26 Statistical Beamformer Statistical ATF modelling Diffused Noise Source Noise Field σ 2 dif ( Φ vv m, m ) ( ) = σdif 2 sinc 2π rm r m σdif 2 λ I. k variance of the diffuse field. For enclosures larger than λ k. Reliability ( 2 σ 2 ) dif R κ,dif (κ 0 ) =1 F η,dif κ 0 α R ξ,dif (ξ 0 ) =1 F η,dif ( 2 α ξ 0 σ 2 d ) η dif χ 2 (2M) Chi-square RV with 2M degrees of freedom. F η,dif (η 0 ) Pr (η dif η 0 ) = γ f (M, η 0 2 ) Γ f (M) is the respective CDF. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

27 Statistical Beamformer Model verification SIR Reliability R,c P=1, empirical P=1. theory P=2, empirical P=2, theory P=3, empirical P=3, theory P=4, empirical P=4, theory c improvement [db] R κ,c M=5, empirical M=5, theoretical M=10, empirical M=10, theoretical M=15, empirical M=15, theoretical M=20, empirical M=20, theoretical M=25, empirical M=25, theoretical κ c improvement (a) M = 5 (b) P = 1 SINR out SNR in for coherent noise sources. T 60 = 0.4sec, room dimensions 4 4 3m. Similar trends for diffused noise field. S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

28 Bibliography References and Further Reading I Erup, L., Gardner, F., and Harris, R. (1993). Interpolation in digital modems, II- implementation and performance. IEEE Transactions on Communications, 41(6): Gannot, S., Burshtein, D., and Weinstein, E. (2001). Signal enhancement using beamforming and nonstationarity with applications to speech. IEEE Transactions on Signal Processing, 49(8): Jacobsen, F. and Roisin, T. (2000). The cohrerence of reverberant sound fields. Journal of the Acoustical Society of America, 108(1): Kodrasi, I., Rohdenburg, T., and Doclo, S. (2011). Microphone position optimization for planar superdirective beamforming. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages , Prague, Czech Republic. Kuttruff, H. (2000). Room acoustics. Taylor & Francis. Liu, Z. (2008). Sound source separation with distributed microphone arrays in the presence of clock synchronization errors. In The International Workshop on Acoustic Echo and Noise Control (IWAENC), Seattle, WA, USA. Lo, Y. (1964). A mathematical theory of antenna arrays with randomly spaced elements. IEEE Transactions on Antennas and Propagation, 12(3): S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

29 Bibliography References and Further Reading II Markovich-Golan, S., Gannot, S., and Cohen, I. (2011). Performance analysis of a randomly spaced wireless microphone array. In The IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages , Prague, Czech Republic. Markovich-Golan, S., Gannot, S., and Cohen, I. (2012). Blind sampling rate offset estimation and compensation in wireless acoustic sensor networks with application to beamforming. In The International Workshop on Acoustic Signal Enhancement (IWAENC), Aachen, Germany. Final list for best student paper award. Markovich-Golan, S., Gannot, S., and Cohen, I. (2013). Performance of the SDW-MWF with randomly located microphones in a reverberant enclosure. IEEE Trans. Audio, Speech and Language Processing. Accepted for publication. Ono, N., Kohno, H., Ito, N., and Sagayama, S. (2009). Blind alignment of asynchronously recorded signals for distributed microphone array. In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pages , New Paltz, NY, USA. Pawig, M., Enzner, G., and Vary, P. (2010). Adaptive sampling rate correction for acoustic echo control in voice-over-ip. IEEE Transactions on Signal Processing, 58(1): Schroeder, M. R. (1987). Statistical parameters of the frequency response curves of large rooms. Journal of the Audio Engineering Society, 35(5): S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30

30 Bibliography References and Further Reading III Wehr, S., Kozintsev, I., Lienhart, R., and Kellermann, W. (2004). Synchronization of acoustic sensors for distributed ad-hoc audio networks and its use for blind source separation. In IEEE Sixth International Symposium on Multimedia Software Engineering, pages S. Gannot (BIU) and A. Bertrand (KUL) Distributed speech enhancement EUSIPCO / 30