ESE 250: Digital Audio Basics. Week 4 February 5, The Frequency Domain. ESE Spring'13 DeHon, Kod, Kadric, Wilson-Shah

Size: px

Start display at page:

Download "ESE 250: Digital Audio Basics. Week 4 February 5, The Frequency Domain. ESE Spring'13 DeHon, Kod, Kadric, Wilson-Shah"

Edmund Fields
5 years ago
Views:

1 ESE 250: Digital Audio Basics Week 4 February 5, 2013 The Frequency Domain 1

2 Course Map 2

3 Musical Representation With this compact notation Could communicate a sound to pianist Much more compact than 44KHz time-sample amplitudes (fewer bits to represent) Represent frequencies 3

4 Physics review Consider a sound signal that varies as a sine wave in time: s( t) Asin( t ) What is the relationship between and f? 2 f f 1/T rad/s, Hz s( t) Asin(2 ft ) 4

5 Time-domain Digital representation Week2: Quantization in TIME and VALUE Question: Other ways to represent this signal? 5

6 Another-domain representation? Consider a sound signal that varies as a sine wave in time: s( t) Asin(2 ft ) Note that a sine wave is periodic, this implies redundancy that we could exploit to better compress it (week3 compression) At least we could save memory by representing only one period. But can we do even better? Question: What information do we need to represent the signal? Amplitude, frequency, phase Question: How much memory to store each of these? 6

7 Another-domain representation? Remember week 2 lab: Inputs: Amplitude: 1 Frequency: various values, 220Hz, etc Phase: 0 Sampling rate: 48,000 khz Output: Periodic sine function sampled in time, with 48,000 samples for 1 second of sound Question: Why did we need to create the large LVM files? Why not just store the little information we started with? 7

8 Storage advantage Assume each of amplitude, frequency and phase are stored as 16bit integers. How much memory to store the sine in lab2? Only storing the info above? With an LVM file containing 48,000 samples? Percent savings? 8

9 Frequency-domain T = π, A = 3: s(t) = A*sin(2π*f*t) = 3*sin(2*t) 9

10 Frequency-domain Of course, not all signals are this simple For example, consider s(t) = sin(2t) + sin(t)/2 Question: What will the frequency representation look like? 10

11 Frequency-domain How about the time-domain? Plot sin(2t) Plot sin(t)/2 Sum: sin(2t)+ sin(t)/2 Notice how it was easier to plot the frequency domain representation 11

12 Frequency-domain Another example What function is this? 12

13 Frequency-domain So far, we have seen how a signal written as a sum of sines of different frequencies can have a frequency domain representation. Each sine component is more or less important depending on its coefficient But can any arbitrary signal be represented as a sum of sines? No. But the idea has potential, let s explore it! 13

14 More background Fourier series: Any periodic signal can be represented as a sum of simple periodic functions: sin and cos sin(nt) and cos(nt) For n = 1, 2, 3, These are called the harmonics of the signal Jean Baptiste Joseph Fourier, wikipedia 14

15 Fourier Series more formally Given a (well-behaved) periodic function f(t), we can always write it as: f N ao ( t) [ an cos( nt) bn sin( nt)] N 0 2 n 1 1 Where: a n f ( t)cos( nt) dt N 0 1 b n f ( t)sin( nt) dt N 1 These two equations give us the an and bn coefficients of the Fourier series based on the input function. These coefficients form the frequency domain representation We will not prove the equations If you re interested: Office Hours, ESE 325, Math 312, 15

16 Fourier Series Why does it work? The cos(nx) and sin(nx) functions form an orthogonal basis: they allow us to represent any periodic signal by taking a linear combination of the basis components without interfering with one another 16

17 Orthogonal basis Choose a basis Order the vectors in it All vectors in the space can be represented as sequences of coordinates, or ordered coefficients of the basis vectors Example: 3D space Basis: [i j k] Linear combination: xi + yj + zk Coordinate representation: [x y z] Example: Quadratic polynomials Basis: [x 2 x 1] Linear combination: ax 2 + bx + c Coordinate representation: [a b c] 17

18 Fourier Series Consider the xy-plane: 10 We can address a point by its two coordinates (x,y) The blue point is located at (10, 3) In Fourier series, we have a basis with infinite dimension, so we sum up over an infinite number of harmonics Harmonics are 1, cos(t), cos(2t), cos(3t),, sin(t), sin(2t),

19 Example of another orthogonal basis Although we have an infinite number of basis elements, we don t need to use all of them The components with the largest coefficients are the most significant The more components we add, the closer to the function we get. i.e.: As, Examples follow N error 0 19

20 Fourier Series Sawtooth wave (falstad.com/fourier/) 20

21 Fourier Series Sawtooth wave (falstad.com/fourier/) 21

22 Interlude 22

23 Where Are We Heading After Today? Week 2 Received signal is discrete-time-stamped quantized q = PCM[ r ] = quant L [Sample Ts [r] ] Week 3 Quantized Signal is Coded c =code[ q ] Week 4 Sampled signal not coded directly First linearly transformed into frequency domain r(t) q Sample Generic Digital Signal Processor Q = DFT[ q ] [Painter & Spanias. Proc.IEEE, 88(4): , 2000] q Code c Psychoacoustic Audio Coder Q Store/ Transmit Decode Produce p(t) c 23

24 What now? In the first half of the lecture we have introduced the idea of frequency domain In the second half of the lecture, we will see how to extend the idea of Fourier Series to the class of signals we are interested in: Not necessarily periodic and quantized 24

25 Frequency-domain Fourier Series deal with periodic signals Are all signals periodic? Is your favorite mp3 song periodic? What do we do with these? 25

26 Frequency-domain Certainly, we could loop a song and make the signal periodic, but this is not a good solution Mathematicians have extended the notion of Fourier Series into that of Fourier Transforms by making the period of the signal go to infinity: T 26

27 Discrete Fourier Transforms Fourier Transforms are nice, but we want to store and process our signals with computers We therefore extend Fourier Transforms into Discrete Fourier Transforms, or DFT The signal is now discrete: x( t) x n The signal contains N samples: 0 n N 1 Remember Euler s formula: e it cos( t) isin( t) DFT ( x DFT n ) X k N 1 n ( X ) x k n N x n e N 1 k 0 i2 kn / N X k e The frequency domain signal X k also has N-1 elements: k i2 kn / N The time domain signal x n can be reconstructed from X k 0 N 1 27

Discrete Fourier Transforms A DFT transforms N samples of a signal in time domain into a (periodic) frequency representation with N samples So we don t have to deal with real signals anymore We

28 Discrete Fourier Transforms A DFT transforms N samples of a signal in time domain into a (periodic) frequency representation with N samples So we don t have to deal with real signals anymore We work with sampled signals (quantized in time), and the frequency representation we get is also quantized in time! DFT ( x n ) X k N 1 n 0 x n e i2 kn / N (sepwww.stanford.edu/oldsep/hale/fftlab.html) 28

29 Discrete Fourier Transforms A smaller sampling period -> More points to represent the signal larger N -> More harmonics used in DFT N harmonics -> Smaller error compared to actual analog signal we capture/produce DFTs are extensively used in practice, since computers can handle them 29

30 Discrete Fourier Transform exercise X DFT ( x k N 1 n 0 x n n ) X k cos( n N 1 n 0 i2 kn / N Complete the table below using the DFT equation shown above x n 2 k ) i N e N 1 n 0 x n sin( n (The bottom is redundant because the DFT of a real signal is symmetrical) 7 samples -> 7 harmonics 2 k ) N 30

knowledge of one harmonic Each harmonic is a component used in the reconstruction of the signal The more

31 Approximating the Sampled Signal Approximate Reconstruction Always achievable A signal sampled in time can be approximated arbitrarily closely from the time-sampled values With a DFT, each sample gives us knowledge of one harmonic Each harmonic is a component used in the reconstruction of the signal The more harmonics we use, the better the reconstruction { Cos[0t], Sin[1t], Cos[1t], Sin[2t], Cos[2t], Sin[3t], Cos[3t] } 31

32 Usually Computed, Not Solved 7 Samples; 7 Harmonics 11 Samples ; 11 Harmonics 15 Samples; 15 Harmonics DFT DFT DFT 32

33 Measured Data: Yet Another Sampled (Real) Signal Sampled Signal: v t ESE 250 Spring 10 DeHon & Koditschek 33

34 Some Signals Dislike Some Harmonics Approximate Reconstruction although always achievable may require a lot of samples to get good performance 15 Samples & Harmonics Sometimes time is better than frequency 21 Samples & Harmonics 31 Samples & Harmonics 34

35 Saving resources However, N can get very large e.g., with a sampling rate of 48,000Hz How big is N for a 4min song? How many operations does this translate to? To compute one frequency component? To compute all N frequency components? This is not practical. Instead, we use a window of values to which we apply the transform Typical size:, 512, 1024, 2048, 35

36 A Window Operation In the example below, we traverse the signal but only look at 64 samples at a time Time Freq (sepwww.stanford.edu/oldsep/hale/fftlab.html) 36

37 A Window Operation The following shows a DFT (on the right) applied to a sinusoidal signal (on the left). The DFT is only applied to part of the signal, the window, which is shown in the middle When we increase the window size, the information of the DFT is more accurate 16-point 32-point 8-point DFT ( 37

38 Moonlight Sonata in Time and Frequency 38

39 Big ideas A signal in time-domain can also be represented in the frequency domain. X(f) = Transform(x(t)) This representation may be more efficient: For storage For signal processing For understanding the wave (e.g. can cut-off some frequencies) We go from a signal represented by values in time to one represented by coefficients of harmonics Fourier Series, for continuous periodic signals FT: Fourier Transform, for continuous signals DFT: Discrete Fourier Transform, for discrete signals DFTs are extensively used in practice Finite amount of data, easy to implement for computers 39

40 Admin Lab 3 due Wednesday (tomorrow) On Thursday, Lab 4: You will be looking at signals in the frequency domain in LabView You will create your own synthesizer! 40

41 Announcement Hackathon and bootcamp If you are interested The bootcamp will be this Saturday, noon to six, in Detkin lab. The Hackathon will begin around 5-6 the weekend of the 15-17th 41

Lecture 5. The Digital Fourier Transform. (Based, in part, on The Scientist and Engineer's Guide to Digital Signal Processing by Steven Smith)

Lecture 5. The Digital Fourier Transform. (Based, in part, on The Scientist and Engineer's Guide to Digital Signal Processing by Steven Smith) Lecture 5 The Digital Fourier Transform (Based, in part, on The Scientist and Engineer's Guide to Digital Signal Processing by Steven Smith) 1 -. 8 -. 6 -. 4 -. 2-1 -. 8 -. 6 -. 4 -. 2 -. 2. 4. 6. 8 1