COMP 546. Lecture 21. Cochlea to brain, Source Localization. Tues. April 3, 2018

Transcription

1 COMP 546 Lecture 21 Cochlea to brain, Source Localization Tues. April 3,

2 Ear pinna auditory canal cochlea outer middle inner 2

3 Eye Ear Lens? Retina? Photoreceptors (light -> chemical) Ganglion cells (spikes)?? Optic nerve? 3

4 Eye Ear Lens Outer ear Retina Cochlea Photoreceptors (light -> chemical) Ganglion cells (spikes) hair cells (mechanical -> chemical) Ganglion cells (spikes) Optic nerve VestibuloCochlear nerve 4

5 Basilar Membrane BM fibres have bandpass frequency mechanical responses. 20,000 Hz 20 Hz 5

6 Basilar Membrane: Place code ( tonotopic ) Nerve cells (hair + ganglion) are distributed along the BM. They have similar bandpass frequency response functions. 20,000 Hz 20 Hz 6

7 Bandpass responses (more details next lecture) ,000 7

8 Neural coding of sound in cochlea Basilar membrane responds by vibrating with sound. Hair cells at each BM location release neurotransmitter that signal BM amplitude at that location Ganglion cells respond to neurotransmitter signals by spiking 8

9 Louder sound within frequency band greater amplitude of BM vibration at that location greater release of neurotransmitter by hair cell greater probability of spike at each peak of filtered wave 9

10 Hair cell neurotransmitter release can signal exact timing of BM amplitude peaks for frequencies up to ~2 khz. For higher frequencies, hair cells encode only the envelope of BM vibrations. t 10

11 Timing of ganglion cell spikes: for frequencies up to 2 KHz ( phase locking ) BM vibration Spikes Hair cells release more neurotransmitter at BM amplitude peaks. Ganglion cells respond to neurotransmitter peaks by spiking. This allows exact timing of BM vibrations to be encoded by spikes. 11

12 Ganglion cells cannot spike faster than 500 times per second. So we need many ganglion cells for each hair cell. 3,000 hair cells in each cochlea (left and right) cochlear nerve (to brain) 30,00 ganglion cells in each cochlea 12

13 Volley code Different ganglion cells at same spatial position on BM 13

14 From cochlea to brain stem BRAIN STEM cochlea cochlear nucleus lateral and medial superior olive (LSO, MSO) auditory cortex 14

15 Tonotopic maps lateral superior olive (LSO) high ω low ω medial superior olive (MSO) auditory nerve cochlear nucleus (CN) cochlea 15

16 Binaural Hearing LSO MSO MSO LSO high ω low ω low ω high ω CN CN 16

17 Binaural Hearing MSO combines low frequency signals. LSO MSO MSO LSO high ω low ω low ω high ω CN CN 17

18 Binaural Hearing LSO combines high frequency signals. LSO MSO MSO LSO high ω high ω CN CN 18

19 Levels of Analysis high - what is the task? what problem is being solved? - brain areas and pathways - neural coding - neural mechanisms low 19

20 For high frequency bands, the head casts a shadow the timing of the peaks cannot be accurately coded by the spikes (only the rate of spikes is informative) For low frequency bands, the head casts a weak shadow only the timing of the peaks can be encoded by spikes 20

21 Duplex theory of binaural hearing (Rayleigh, 1907) level differences computed for higher frequencies (ILD -- interaural level differences) timing differences computed for lower frequencies (ITD - interaural timing differences) 21

22 Level differences (high frequencies) Excitatory input comes from the ear on the same side. Inhibitory input comes from ear on the opposite side. high ω LSO MSO MSO + + LSO high ω CN CN 22

23 Timing differences (low frequencies) Sum excitatory input from both sides. Reminiscent of binocular complex cells in V1? LSO MSO MSO LSO high ω CN + low ω + + low ω + CN high ω 23

24 Jeffress Model (1948) for timing differences from left ear E D C B A from right ear C B D A E 24

25 Spike Timing precision required for Jeffress Model? from left ear E D C B A from right ear distance = 1 10 millimetres speed of spike = 10 metres second 1 time = distance speed = millisecond See Exercises 19 Q2c 25

26 Jeffress model remains controversial. It is not known exactly how coincidence detection occurs in MSO. ω A B C D E Coincidence detection for each low frequency band A B C D E A B C D E A B C D E 26

27 Naïve Computational Model of Source Localization (Recall lecture 20) I l (t) = α I r (t τ) + ε(t) shadow delay model error Find the α and τ that minimize T t=1 { I l (t) α I r (t τ) } 2 where τ < 0.5 ms. 27

28 Timing difference: find the τ that maximizes t I l (t) I r (t τ). Level difference: T t=1 I l (t) 2 10 log 10 T I r (t) 2 t=1 28

29 For each low frequency band j, find the τ that maximizes t I left j (t) I right j (t τ). (or use summation model similar to binocular cells or Jeffress model) An estimated value of delay τ in frequency band j is consistent with various possible source directions ( φ, θ ). Similar to cone of confusion, but more general because of frequency dependence 29

30 For each high frequency band j, compute interaural level difference (ILD) : T t=1 I j left (t) 2 ILD j = 10 log 10 T t=1 I j right (t) 2 What does each ILD j tell us? 30

31 Recall head related impulse response function (HRIR) from last lecture.. If the source direction is (q, f), and g j t is the filter for band j. then j I left t; φ, θ = g j t h left (t; φ, θ) I src t; φ, θ j I right t; φ, θ = g j t h right (t; φ, θ) I src t; φ, θ 31

32 Take the Fourier transform and apply convolution theorem : j I left ω; φ, θ = g j ω h left (ω; φ, θ) I src ω; φ, θ j I right ω; φ, θ = g j ω h right (ω; φ, θ) I src ω; φ, θ 32

33 Take the Fourier transform and apply convolution theorem : j I left ω; φ, θ = g j ω h left (ω; φ, θ) I src ω; φ, θ j I right ω; φ, θ = g j ω h right (ω; φ, θ) I src ω; φ, θ If there is just one source direction (φ, θ), then for each frequency ω within band j j I left j I right ω h left (ω; φ, θ) ω h right (ω; φ, θ) 33

34 One can show using Parseval s theorem of Fourier transforms that if h left (ω; φ, θ) and h right (ω; φ, θ) are approximately constant within band j, then: T t=1 I j left (t) 2 j h left ( φ, θ) 2 T t=1 I j right (t) 2 j h right ( φ, θ) 2 34

35 One can show using Parseval s theorem of Fourier transforms that if h left (ω; φ, θ) and h right (ω; φ, θ) are approximately constant within band j, then: T t=1 I j left (t) 2 j h left ( φ, θ) 2 T t=1 I j right (t) 2 j h right ( φ, θ) 2 The ear can measure this and can infer source directions ( φ, θ) that are consistent with it. 35

36 Interaural Level Difference (db) as a function of (φ, θ) for two fixed ω. 700 Hz 11,000 Hz Each iso-contour in each frequency band is consistent with a measured level difference (db). 36

37 Monaural spectral cues (Spatial localization with one ear?) I j t; φ, θ = g j t h(t; φ, θ) I src t; φ, θ I j ω; φ, θ = g j ω h j (ω; φ, θ) I src ω; φ, θ Pattern of peaks and notches across bands will be due to HRTF, not to the source. If the source is noise, then all frequencies make the same contribution on average. 37

38 Pinnal notch frequency varies with elevation of source e.g. in the medial plane. I j ω; φ, θ = g j ω h j (ω; φ, θ) I src ω; φ, θ 0 0 Azimuth θ = 0 HRTF from last lecture e.g. medial plane 38

39 Levels of Analysis high - what is the task? what problem is being solved? Source localization using level and timing differences within frequency channels. - brain areas and pathways (cochlea to CN to MSO and LSO in the brainstem) - neural coding (gave sketch only) - neural mechanisms low 39