Source localization and separation for binaural hearing aids

Transcription

1 Source localization and separation for binaural hearing aids Mehdi Zohourian, Gerald Enzner, Rainer Martin Listen Workshop, July 218 Institute of Communication Acoustics

2 Outline 1 Introduction 2 Binaural speaker localization 3 Multichannel speaker separation 4 Evaluation results 5 Conclusion

3 Binaural speech enhancement Hearing aids (HAs) Challenges Ambient noise and interference Source not necessarily in front of the listener Significant head movements Solutions Binaural localization using joint IPD/ILD Tracking the listener s head turns Integration into adaptive binaural beamformer i ence noise noise t g i noise ence t g noise i ence noise Proposing binaural localization algorithms using cost functions based on beamforming and statistical model-based techniques Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 2 / 12

4 Binaural signal model in the STFT domain S q S q θ q Y 1 Y 2 π 2 π 2 Y 3 Y 4 Y (k,µ) = H(k,Θ)S(k,µ)+V (k,µ) (1) H: matrix of binaural room transfer functions (BRTFs) (k,µ): frequency and frame indices Θ = [θ 1,...,θ q,...,θ Q ]: DOAs Spectral disjointness leads to one dominant source at each frequency bin Y (k,µ) = H(k,θ q )S q (k,µ)+v (k,µ) (2) Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 3 / 12

5 Cost functions for binaural localization S q (k) H R2 (k,θ q) H R1 (k,θ q) θ q H L1 (k,θ q) H L2 (k,θ q) Y R (k) Y L (k) V L (k) V R (k) J(k,θ) A grid search in steps of e.g., 5 Target beamforming (TBF) Null-steering beamforming (NBF) Deterministic maximum likelihood (DML) Stochastic maximum likelihood (SML) Narrowband DOA estimation: ˆθ(k,µ) = argmax(j(k,θ)) Broadband DOA estimation: ˆθ(µ) = argmax θ θ ( K k=1 J(k,θ) ) Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 4 / 12

6 Binaural localization algorithms [Zohourian, Enzner, Martin 218] Beamforming-based using filter-and-sum beamformer (W ) TBF uses energy-normalized matched-filter NBF uses energy-normalized filter using cross-relation technique Model-based methods using maximum likelihood optimization The noise samples follow zero-mean Gaussian distribution The source signal is unknown and deterministic DML The source signal is a stochastic process [Yee/Degroat99] SML Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 5 / 12

7 Adaptive beamformer Generalized side-lobe canceller (GSC) W fq (k,µ): Fixed beamformer Y 1 (k,µ) Y M (k,µ) W fq (k,µ) S q (k,µ) Ŝ q (k,µ) B q (k,µ): Blocking matrix W Vq (k,µ): Noise canceller B q (k,µ) W Vq (k,µ) [Griffiths and Jim, 1982] Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 6 / 12

8 Adaptive binaural beamformer for source separation under head turns Extension to 2 2 binaural HA microphones Model-based binaural GSC Binaural beamformer (MVDR) Binaural DOA estimation TBF, NBF, SML, DML Target presence probability using Gaussian mixture model [Madhu and Martin, 211] Y 1 (k,µ) S q (k,µ) W fq (k,µ) Y M (k,µ) ˆθ ˆθ(µ) DOA EM SqˆΘ ˆθ(k, µ) B q (k,µ) W Vq (k,µ) TPP Ŝ q (k,µ) p(h θ S q (k,µ) ˆθ(k,µ)) = [Zohourian, Enzner and Martin, 218] ρ Sq N (ˆθ(k,µ) µsq,σsq) 2 ) Q i=1 ρ S i N (ˆθ(k,µ) µsi,σs 2 i Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 7 / 12

9 Experimental setup and evaluation Data 2 2 BTE hearing aids attached to a dummy head Loudspeakers playing speech signals, 1.2 m distance A reverberated room with T 6 =.4s Various background noise Sampling rate=16 khz, DFT size= 124, frame advance= 32 ms HRTF prototypes from the database [Kayser et al. 29] Evaluation criteria Localization accuracy using anomalies: percentage of frames with localization error more than 1 Adaptive binaural beamformer performance using PESQ, STOI, SIR Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 8 / 12

10 Anomalies Anomalies Broadband localization for static scenarios Results with two talkers averaged across 11 mixtures one talker at 3, other at, 3, ±6, ±9, ±12, ±15, Broadband DOA estimation NBF TBF SML DML db (a) No noise Broadband DOA estimation NBF TBF SML DML 1dB (b) Spatially uncorrelated white noise Anomalies Anomalies Broadband DOA estimation NBF TBF SML DML db (c) Spatially diffuse white noise Broadband DOA estimation 1dB NBF TBF SML DML db 1dB (d) Spatially diffuse babble noise SML is more robust in broadband DOA estimation than other methods Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 9 / 12

11 Evaluation under listener head turns Spatially diffuse babble noise with 1dB SNR source position ( ) NBF TBF SML DML Time (sec) (a) Broadband localization RMSE (Radian).2.1 NBF TBF SML DML (b) RMSE Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 1 / 12

12 Evaluation under listener head turns Spatially diffuse babble noise with 1dB SNR NBF 1 1 source position ( ) RMSE (Radian) TBF SML DML Time (sec) (a) Broadband localization NBF TBF SML DML PESQ STOI SIR NoNoise UP NBF TBF SML DML Uncorr DiffWhite DiffBabble (b) RMSE Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 1 / 12

13 Audio demonstration S 1 π 6 S 1 S 2 π 6 π 6 π 2 π 2 π 2 π 2 S 2 π 6 π π π 6 6 S 3 S π 4 Two sources (at 3 and 15 degrees): mic signal Play Four sources (at ±3 and ±15 degrees): mic signal Play source 1 Play source 2 Play source 1 Play source 3 Play source 2 Play source 4 Play Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 11 / 12

14 Summary and conclusion Summary Beamforming-based and statistical model-based binaural localization GSC-based binaural speaker separation SML-based adaptive binaural beamforming Conclusion SML is robust in noisy/dynamic environment but computationally complex TBF is promising in static scenarios and for narrowband localization and less complex SML-based GSC is a good candidate in noisy/dynamic environment Robust against reverberation and mismatch of HRTFs Thank you for your attention! Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 12 / 12

15 Influence of the mismatch of HRTFs Two cases: Binaural signals are rendered using the HRTF database recorded in our anechoic room The same database is used as HRTF prototypes for localization UP NBF TBF SML PESQ Robust against the mismatch of HRTFs STOI SIR HRTF-matched HRTF-mismatched In case of mismatch of the head size adaptive binaural localization approach based on the joint optimization of head radius and DOA could be useful Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 13 / 12

16 Cost functions for binaural localization methods TBF cost functions J(k,θ) H H (k,θ)ˆφ Y Y (k)h(k,θ) H H (k,θ)h(k,θ)y H (k)y (k) NBF H W NBF(k,θ)Φ (m,m ) Y Y (k) W NBF(k,θ) Y m(k) Y m,m m (k) DML H H (k,θ)γ 1 V V (k)ˆφ Y Y (k)γ 1 V V (k)h(k,θ) H H (k,θ)γ 1 V V (k)h(k,θ)y H (k)y (k) SML logdet{p H(θ)ˆΦ Y Y (k)p H H(θ)+ ˆΦ V V P H(θ)Γ V V (k)} Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 14 / 12

17 Binaural localization algorithms [Zohourian, Enzner, Martin 218] Beamforming-based using filter-and-sum beamformer (W ) TBF uses energy-normalized matched-filter W TBF (k,θ) = H(k,θ) H(k,θ) NBF uses energy-normalized filter using cross-relation technique 1 W NBF (k,θ) = Hm(k,θ) 2 + H m (k,θ) 2[H m (k,θ), H m(k,θ)] H Model-based methods using maximum likelihood optimization The noise samples follow zero-mean Gaussian distribution The source signal is unknown and deterministic DML 1 P(Y θ,φ V V ) = π M Φ V V exp( (Y HS) H Φ 1 V V (Y HS)) The source signal is a stochastic process [Yee/Degroat99] SML ( ) 1 P(Y θ,φ Y Y ) = π M Φ Y Y exp Y H Φ 1 Y Y Y Φ Y Y = HH H Φ SS +Φ V V (3) Introduction Binaural localization Speaker separation Evaluation Conclusion M. Zohourian 15 / 12