Topic 7. Convolution, Filters, Correlation, Representation. Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio

Transcription

1 Topic 7 Convolution, Filters, Correlation, Representation

2 Short time Fourier Transform Break signal into windows Calculate DFT of each window

3 The Spectrogram spectrogram(y,1024,512,1024,fs,'yaxis'); A series of short term DFTs Typically just displays the magnitudes of X from 0 Hz to Nyquist rate

4 Equal Temperament Octave is a relationship by power of 2. There are 12 half-steps in an octave n number of half-steps from the reference pitch frequency of desired pitch f 2 12 f ref frequency of the reference pitch

5 Spiral Pitch representation

6 frequency Chroma: Many to one Chroma = log2(freq) floor(log2(freq)) Chroma periodic in range 0 to (almost) 1 Chroma map on to pitch classes Hz Hz 0.75 CHROMA Hz 100 Hz time 0.5

7 Making a Chromagram Decide how to quantize (bin) the chroma range. 12 pitch classes? 120 bins? Equal temperment? Make a spectrogram For each time-step in the spectrogram find the chroma for each frequency from 0 to N/2 Sum the amplitude of all frequencies with the same chroma bin (Some chromagrams also add in the energy from the odd harmonics) Place that value in the chroma bin

8 Chromagram of Clarinet C C# 2 D D# E F F# G G# A A# B

9 Chromagram of Clarinet

10 Convolution convolution is a mathematical operator which takes two functions f and g and produces a third function that represents the amount of overlap between f and a reversed and translated version of g. Typically, one of the functions is taken to be a fixed filter impulse response, and is known as a kernel. b ( f g)( t) f ( ) g( t ) d Convolution operator a

11 Discrete Convolution convolution is a mathematical operator which takes two functions f and g and produces a third function that represents the amount of overlap between f and a reversed and translated version of g. Typically, one of the functions is taken to be a fixed filter impulse response, and is known as a kernel. ( f g)[ m] f [ n] g[ m n] n

12 Convolution (Time Domain) function C= convolution(a,b) lengtha= length(a); lengthb= length(b); C = zeros(1, lengtha + lengthb - 1); for m = 1:lengthA TIMES! Not Convolution for n = 1:lengthB C(m+n-1) = C(m+n-1) + A(m)*B(n); end end

13 Convolution (Frequency Domain) Convolution is defined in the time domain as and in frequency domain as.. y( t) x( t)* h( t) y( t) x( ) h( t ) d Y( ) X ( ) H ( )

14 Speeding implementation How long does convolution take? Think about these facts FFT(x) takes O(nlog(n)) time y( t) x( t)* h( t) Y( ) X ( ) H( ) Knowing this, how can we speed convolution?

15 Convolution = Filtering (in the time domain) Source Signal x(t) Filter h(t) Output Signal y(t) x( t)* h( t) y( t) Convolution

16 So what? So now you can design a quick-n-dirty bandpass filter H(k)! Steps Decide on the frequency resolution (number of points) Set the amplitudes of the different frequencies as you like Take the inverse FFT of your filter Do convolution with your signal Let s try it!

17 Impulse response An impulse is this signal: () t 1 if t 0 0 else The impulse response h(t) of a filter is what happens when you do convolution of the impulse signal with the filter. The impulse response h(t) of a filter is the same as the IFFT of the transfer function H( ) of that filter.

18 Multiplication = Filtering (in the frequency domain) Source Signal X( ) Filter H( ) Output Signal Y( ) X ( ) H( ) Y( ) Multiplication Transfer function

19 Multiplication (in the frequency domain) X ( ) H( ) X[ k] H[ k] Multiply kth frequency of X by the kth frequency of H This is COMPLEX multiplication Doing X.*H in Matlab takes care of this

20 Estimating H(k) (in the frequency domain) X[ k] H[ k] Y[ k] so... H[ k] Y[ k]/ X[ k]...more or less.

21 Noise and Distortion Source Signal X( ) Filter H( ) Distortion D( ) + Output Signal Y( ) Noise N( ) Distortion and noise make it a lot harder to estimate H

22 Estimating H Hope noise and distortion are not correlated with X. Call noise + distortion N. Then X[ k] H[ k] N[ k] Y[ k]

23 Estimating H Estimate H a lot of times, in the hopes that the noise will wash out in the mix Hˆ [ k ] Y [ k ] X [ k ] / X [ k ] X [ k ] where N Y k X k Y k X k n n N n 1 1 [ ] [ ] [ ] [ ] The average value Over N estimates

24 Caveats This really only works on the frequencies where there was energy in the input signal X. If there wasn t energy well then we re out of luck. So it is best to use broadband white noise for X, if you can help it.