Lecture 6 January 21, 2016

Transcription

1 MATH 6/CME 37: Applied Fourier Analysis and Winter 06 Elements of Modern Signal Processing Lecture 6 January, 06 Prof. Emmanuel Candes Scribe: Carlos A. Sing-Long, Edited by E. Bates Outline Agenda: Fourier series. Aliasing example. Discrete convolutions 3. Fourier series 4. Another look at Shannon s sampling theorem Last Time: We introduced the Dirac comb and proved the Poisson summation formula, which states that the Fourier transform of a Dirac comb of period T is also a Dirac comb, but with period π/t (modulo a mutiplicative factor π/t ). We discussed the problem of sampling an analog signal f(t) by measuring its values at time intervals of length T, and we represented the discretized signal f d as a superposition of Dirac delta functions. The Poisson summation formula was instrumental in proving the aliasing formula, which states that the Fourier transform ˆf d of the discretized signal is the π/t -periodization of the spectrum ˆf of the analog signal. From this calculation, we saw that when f is bandlimited and T is sufficiently large, no aliasing occurs in the periodization. When this is the case, we can recover the original signal from its samples via Shannon s sampling theorem. Today: We will first revisit the topic of aliasing by observing it for an actual signal. Then, in the pursuit of further studying discrete signals, we shall introduce Fourier series. Understanding these discrete sums as a special case of the Fourier integrals we know well, we will survey the properties Fourier series have in analogy with Fourier transforms. In particular, the Poisson summation formula will be employed once more to show that complex exponentials again form an orthonormal basis of L -space. The machinery of this basis will actually provide a second proof of Shannon interpolation. A case of aliasing To make concrete our study of aliasing, let us consider the very simple signal ( ) 3π f(t) = cos t = [e i 3π t + e i 3π t].

2 Its Fourier transform is ˆf(ω) = [ ( δ ω + 3π ) ( + δ ω 3π )], meaning f is bandlimited on [ 3π, 3π ]. From last lecture we know that the Nyquist rate is then 3 (three samples for every two units of time). Suppose we did not know this and instead guessed f were bandlimited on [, π]. We would then believe samples were only needed every unit of time. But this sampling rate is now below Nyquist and thus too slow to prevent aliasing. Let us see why: The Fourier transform of the discretization f d = n Z f(n)δ n with T = is ˆf d (ω) = ˆf(ω πk) = δ (ω + 3π ) πk + δ, (ω 3π ) πk. () which is a spike train of spacing π. What we then see in the frequency range [, π] is not ˆf, but rather ˆf d (ω)i{ ω π} = ĝ(ω) = [ ( δ ω + π ) ( + δ ω π ) ], }{{ }}{{ } from k = from k = k = k =0 k = k = k =0 k = Figure : Shown above is the portion of the Dirac comb ˆf d in [, π], which has contributions from five different copies of ˆf, corresponding to k = 0, ±, and ± in (). Part of each of the k = ± copies is aliased into the interval [ 3π, 3π ], giving artifact delta functions (displayed with solid lines) in the supposed frequency band [, π]. Such aliasing produces low-frequency artifacts in signal reconstruction. This is the Fourier transform of ( π ) g(t) = cos t, a cosine wave with frequency three times smaller than that of f. Indeed, if Shannon s formula is naively applied to ˆf d, the signal recovered is f(n) sinc(t) = ( ) 3π cos n sinc(t) = ( π ) cos n sinc(t) = g(n) sinc(t) = g(t) n Z n Z n Z n Z

3 since g is bandlimited to [, π] and f is not. In other words, by sampling f at too low a rate, we cannot detect the difference between the signal f and the signal g: their values at the integers are identical. Shannon interpolation can only return the unique function that gives those samples and is bandlimited on [π, π]. In this case, that function is g(t). Figure : When sampled at the integers, the functions f(t) = cos ( 3π t) and g(t) = cos ( π t) return identical values. The higher frequency of f (solid, black) is aliased into the lower frequency band [, π], giving a reconstruction of g rather than of f. 3 Discrete convolutions Now we consider discrete signals {f(n)} n Z, or more precisely, discrete-time signals. As in lecture, we are interested in linear maps L : f Lf. We define for τ Z the translation of f as f τ (n) = f(n τ), n Z, and we say L is time-invariant if (Lf) τ = Lf τ. The discrete impulse δ is defined as { n = 0, δ(n) = 0 otherwise, and the discrete convolution between f and g as (f g)(n) = m Z f(m)g(n m). As before, δ acts as the identity under convolution: (f δ)(n) = m Z f(m)δ(n m) = f(n)δ(0) = f(n), 3

4 Consequently, for any time-invariant map L, (Lf)(n) = f(m)(lδ m )(n) = f(m)(lδ) m (n) = f(m)(lδ)(n m). m Z m Z m Z If we let h := Lδ, we see that Lf = f h. The kernel h is called the impulse response for L, and we conclude that every time-invariant map can be written as the convolution with the impulse response. It is easy to check that the converse is true, and so a map is time-invariant if and only if it is a convolution with a fixed function. We saw in the continuous-time case that complex exponentials are eigenvectors of time-invariant operators. The same is true in the discrete setting, but now the exponentials are evaluated only at integers. Indeed, write e ω (n) = e iωn, n Z. For a time-invariant map L, (Le ω )(n) = (h e ω )(n) = m Z h(k)e iω(n m) = e iωn m Z h(m)e iωm = e ω (n) m Z h(m)e ω (m). We once more have a transfer function ĥ(ω) := h, e ω = m Z h(m)e ω (m) = m Z h(m)e iωm, defined to be return the eigenvalue for the eigenfunction e ω : Le ω = ĥ(ω)e ω. We call ĥ the Fourier series of h. In fact, this is just a special case of the Fourier transform. Considering the discretization f d (t) = f(n)δ(t n), n Z of a continuous-time signal f(t), we have ˆf d (ω) = f(n)e iωn, n Z which is the Fourier series of the discrete-time signal {f(n)} n Z. 4 Fourier series What separates Fourier series from Fourier transforms is their periodicity. Since only integer values of m are considered, ĥ(ω) = h(m)e iωm m Z is necessarily π-periodic in ω. This fact is actually one we already know, since we learned in Lecture 5 that discretization in time is equivalent to periodization in frequency. Obviously Fourier transforms need not be periodic. After all, the inverse Fourier transform told us that every L function is a superposition of complex exponentials with frequencies in R. Once 4

5 again drawing analogy with the continuous case, we can ask: Is every periodic L function (with period π) the superposition of exponentials with frequencies in Z? Perhaps less surprising now, but no less amazing, the answer is yes. To properly develop this answer, we consider a periodic domain S = R/πZ (read R modulo πz ), which we call the circle. To say f is a function on S just means f is π-periodic: f(t) = f(t + π) for every t R. Instead of working in the vector space L (R), we now work in L (S), the vector space of squareintegrable functions on the circle: { } L (S) = f : S C : f(t) dt <. L (S) is a Hilbert space under the inner product f, g = f(t)g(t) dt, and complex exponentials with integer frequencies form a basis. Theorem. The sequence {e k } where is a complete orthonormal set for L (S). e k (t) = π e ikt, Recall from linear algebra that if {v, v,..., v n } is an orthonormal basis of an n-dimensional inner product space, any vector v has representation n v = v k, v v k. k= The analogous statement is true for infinite-dimensional Hilbert spaces, namely that if {v k } is a complete orthonormal set, then v = v k, v v k. () So Theorem states that for f square-integrable in [, π] with f() = f(π), we can write f = c k e k with c k = e k, f, (3) which means f(t) = c k e ikt with c k = π π f(t)e ikt dt. Here the right-hand side of the first expression is the Fourier series of f, and we understand the equality to hold in the L sense. That is, f(t) is equal to π c ke ikt for all t outside a set of measure zero. Equivalently, f(t) is the L -limit of the partial sums S n f(t) = c k e ikt π as n. The coefficients c k = e k, f are called the Fourier coefficients of f. 5

6 4. The Parseval-Plancherel theorem Before proving Theorem, we make some linear algebraic observations. Not only do orthonormal bases provide simple vector representations, but they also lend themselves to easy computations of inner products, in particular norms: If v is given by (), then v = v, v = j, v v j, j Z v v k, v v k = v j, v v k, v δ jk = v k, v. j, So even without Theorem, whenever f = e k, f e k g = k, g e k e we must have f, g = e k, f e k, g and f = e k, f. The inner product on the left is occurring in L (S), while the inner product on the right is taking place in l, the space of square-summable sequences. This duality is the Fourier series analog of the Parseval-Plancherel theorem for Fourier transforms. We may even write the latter equality as f = π ˆf(k), where ˆf(k) is now given by an integral not over R, but rather over S: ˆf(k) = f(t)e ikt dt. Exercise: Use the Parseval-Plancherel identities to prove k= k = π 6. Hint: Consider the Heaviside function f(t) = I{t > 0}. 4. Proof of Fourier series L -convergence We will now prove that complex exponentials with integer frequencies form an orthonormal basis for L (S). Our observations regarding the Parseval-Plancherel identities will be useful. In particular, they imply Bessel s inequality: e k, g g g L (S), A Z. k A Proof of Theorem. A direct computation, which we omit for the sake of brevity, shows that {e k } is an orthonormal set. To prove its completeness, we will first show that any φ C (S) is expressible in the form (3), and then appeal to a density argument. The details of the density 6

7 argument are technical and tangential to the key ideas of the proof; the only necessary fact is that C functions are dense in L. Now, for any φ C (S), we have S n φ(t) = e k, φ e k (t) = π φ(u)e ik(t u) du = π φ(u)e ik(t u) du. Recall from our proof of the Poisson summation formula that the distribution e ik( ), when restricted to [, π], is equal to πδ. Conveniently, periodicity allows us to integrate over any interval of length π, so let s choose [, π]! In the limit as n, we can evaluate the last integral above as π φ(u)e ik(t u) du = t t π φ(u + t)e iku du = Indeed, we have shown pointwise convergence S n φ φ as n. π φ(u + t)e iku du = φ(t). To show that this convergence is uniform, first note that φ, φ L (S). For k 0, integration by parts gives e k, φ = π φ(t)e ikt dt = ik e k, φ, which implies via Cauchy-Schwarz and Bessel s inequality e k, φ = e 0, φ + k e k, φ 0< e 0, φ + <. φ + φ k 0< π 3 / 0< e k, φ Consequently, the sequence { e k, φ } is absolutely summable. Since e k (t) is uniformly bounded by, the Weierstrass M-test implies the partial sums converge uniformly to a continuous function. Since we know this limit is φ, we conclude that S n φ converges pointwise uniformly to φ. Furthermore, we immediately obtain L convergence: φ S n φ π φ S n φ 0 as n. We now complete the proof by appealing to density. Let f L (S) and ε > 0 be given. For any n and any φ C (S), we have S n f S n φ = S n (f φ) = e k, f φ f φ. By density, we can choose φ C (S) and N sufficiently large that f φ < 3 ε and φ S nφ < 3 ε n N. / 7

8 An application of the triangle inequality yields proving the claim. f S n f f φ + φ S n φ + S n φ S n f < ε n N, 5 Another view at Shannon s sampling theorem For a neat application of our Fourier series machinery, let us conclude with a second derivation of Shannon s sampling theorem. Suppose f(t) L (R) (so ˆf is also in L (R)) is bandlimited to [, π]. By changing the order of summation, we can write ˆf(ω) = n Z e n, ˆf e n (ω), for ω [, π]. If we let ĥ(ω) = I{ ω π} be the Fourier transform of the sinc function, then we can further write ˆf(ω) = n Z e n, ˆf e n (ω)ĥ(ω), for all ω R. Since the previous computation becomes e n, ˆf = π e iωn ˆf(ω) dω = πf(n), ˆf(ω) = n Z f(n)e iωn ĥ(ω), which is exactly Shannon s sampling theorem: taking inverse Fourier transform yields f(t) = n Z f(n)h(t n). In this way, we see that Shannon s sampling theorem is a consequence of the completeness of the complex exponentials over L (S). 8