MATH 262/CME 372: Applied Fourier Analysis and Winter 26 Elements of Modern Signal Processing Lecture January 5, 26 Prof. Emmanuel Candes Scribe: Carlos A. Sing-Long; Edited by E. Candes & E. Bates Outline Agenda: Fourier Integrals. Time-invariant operators 2. Convolutions 3. Continuous-time Fourier transform 4. Fourier transforms of simple functions 2 Time-invariant linear operators Here we consider signals f = f(t) with t T where typically T = Z or R. We will be interested in linear mappings f g = Lf. This map can be represented in discrete time as g(t) = u Z K(t, u)f(u), and in continuous time as g(t) = R K(t, u)f(u) du. Without fear of confusion, we will drop the R in the integral above. An important example is the time-shift operator f f τ where f τ (t) = f(t τ). An operator L is called a time-invariant operator (TIO) if g τ = (Lf) τ is equal to Lf τ. In other words, Input delayed by τ = Output delayed by τ. Examples: Electrical circuits Linear filtering Derivatives (Lf = f )
3 Convolutions The convolution product in R is defined as (f g)(t) = f(u)g(t u) du. The convolution product is (i) Commutative: f g = g f for any f, g. (ii) Associative: (f g) h = f (g h) for any f, g, h. For fixed h we can define the convolution operator f g = h f. Roughly speaking, the output g is the local averages of the input f, weighted by the function h. For instance, when h is a Gaussian curve, g is a smoothed version of f. A convolution operator is a TIO. Indeed, (Lf τ )(t) = = R R L f(u τ)h(t u) du f(u)h((t τ) u) du = (f h)(t τ) = (Lf) τ (t). Conversely, every TIO is a convolution operator. To see this, we write g(t) = h(t, u)f(u) du. This is the continuous analog of a matrix-vector product. To prove the claim, consider on one hand g τ (t) = (Lf) τ (t) = h(t τ, u)f(u) du. () On the other, consider (Lf τ )(t) = h(t, u)f τ (u) du = h(t, u)f(u τ) du = h(t, u + τ)f(u) du. (2) Since L is a TIO, g τ = Lf τ and so () and (2) yield h(t τ, u)f(u) du = h(t, u + τ)f(u) du = (h(t τ, u) h(t, u + τ))f(u) du, for any f. In turn h(t, u + τ) = h(t τ, u) for t R. Consequently h(t, u) = h( ( t), u) = h(, u t) = h(t u) 2
is a function of only the difference t u. We now see that L is a convolution operator (Lf)(t) = h(t u)f(u) du. (3) In (3), h is referred to as the convolution kernel. In application, this kernel is often called the impulse-response function, for reasons we will see below. In optics and imaging, h is also called the point spread function (PSF). When observing a very localized and bright spot, similar to a physical realization of an impulse, we observe a blurred source; this blurred image is the convolution kernel of the system (c.f. Airy disk). 3. Response to impulse (a) T = Z: In this domain, the impulse is the function δ defined as { t =, δ(t) = t. We have and f, δ = u Z f(u)δ(u) = f(), (f δ)(t) = u Z f(u)δ(t u) = f(t), from which it follows that f δ = f. That is, the impulse is the identity of the convolution product. When we apply a TIO L to the impulse, we obtain Lδ = h δ = h, meaning the convolution kernel h is the response to the impulse δ. The name impulse-repsonse is thus fitting. (b) T = R: In this case the impulse is the Dirac delta function δ, which is better called a distribution or generalized function. In analog to the discrete case, we define f, δ = f(u)δ(u) du = f(), and (f δ)(t) = f(u)δ(t u) du = f(t). In the language of probability, then, δ is a point mass of weight at t =. Applying a TIO L to the impulse yields Lδ = h δ = h, and once again we see the convolution kernel is the response to the impulse δ. 3
3.2 Transfer functions When a TIO acts on a complex exponential f(t) = e iωt it has the form Lf(t) = h(u) e iω(t u) du = e iωt h(u) e iωu du = ĥ(ω) eiωt. }{{} This is a very simple action! In fact, the TIO acts as multiplication. This says that complex exponentials are eigenvectors of TIOs with eigenvalues ĥ(ω) = h(u) e iωu du. (4) ĥ(ω) The function ĥ of frequency ω is called the transfer function. Notice that ĥ(ω) is just the inner (dot) product of h with the trigonometric exponential function e iωu. Given how easy it is to understand the action of TIOs on complex exponentials, it is tempting to decompose any signal f as a superposition of these eigenvectors. Amazingly, Fourier analysis shows that this is possible! 3.3 Causal filter A TIO is said to be causal if Lf(t) does not depend of the values f(u) for u > t. From this we deduce that g(t) = h(u)f(t u) du = h(u) = for u <, that is, the transfer function of a causal TIO vanishes over negative numbers. 4 Fourier integrals Motivated by the form of the transfer function (4), we make the following definition: Definition. The Fourier transform of a function f L (R) is defined as ˆf(ω) = f(t) e iωt dt. Note: There are other possible normalizations. For instance, one could instead define ˆf(ω) = f(t) e 2πiωt dt. Because f L (R), the integral converges to make ˆf well-defined: ˆf(ω) = f(t)e iωt dt f(t)e iωt dt = f(t) dt <. 4
.5.5.5 2 2 f(t).5 8 4 4 8 ˆf(ω) Figure : Fourier transform pair for the boxcar function Exercise: Show that ˆf is continuous as a function of ω. 4. Fourier integrals of simple functions Let us compute the Fourier transforms of some common signals. (i) Boxcar: By direct computation when ω, ˆf(ω) = /2 /2 f(t) = e iωt dt = { t [ 2, 2 ], otherwise. iω e iωt /2 /2 = sin(ω/2). ω/2 And when ω =, ˆf() = /2 /2 dt =. We can express this result in terms of a useful function we will see many times during this course: the sinc function is given by { sin(πt) sinc(t) = πt, t,, t =. In this notation, we have ˆf(ω) = sinc(ω/2π). 5
.5.5.5 4 2 2 4 f(t).5 8 4 4 8 ˆf(ω) Figure 2: Fourier transform pair for the triangle function (ii) Triangle: f(t) = { t t [, ], otherwise. Once again by direct computation, when ω, And ˆf() = = sinc 2 (). ˆf(ω) = = 2 f(t) cos(ωt) dt ( t) cos(ωt) dt = 2( t) sin(ωt) ω = 2 ( cos ω) ω2 = 4 sin(ω/2)2 ω 2 = sinc 2 (ω/2π). + 2 ω sin(ωt) dt (iii) Gaussian bell curve: f(t) = 2π e t2 /2 Note that R f(t) dt =, making f(t) a valid probability density. Its Fourier transform is ˆf(ω) = e ω2 /2. Here we outline two ways to obtain it. 6
Complex variable arguments: We can extend f as an analytic function f : C C with f(z) = e z2 /2. Therefore the integral over the contour Γ (see Fig. 4) is such that f(z) dz =, and Γ f(z) dz = ( Γ + Γ + Γ + Γ ) Γ f(z) dz = L L ( L ) f(t) dt f(t+iω) dt+ + f(z) dz. L Γ Γ By enlarging Γ so that Γ and Γ go to infinity (that is, taking the limit L ) the corresponding integrals tend to zero (since f(a + bi) = e (a+bi)2 /2 = e b2 a 2 tends to zero as a ± and b remains fixed). Then using the fact that f(t + iω) = f(t) e ω2 /2 e iωt, we obtain ( ) = f(t) dt e ω2 /2 f(t) e iωt dt = ˆf(ω) = f(t) dt e ω2 /2 = e ω2 /2. A differential equation: A direct computation shows that ˆf(ω) = 2π e t2 /2 e iωt dt and ˆf (ω) = 2π = ω 2π ( it) e t2 /2 e iωt dt e t2 /2 e iωt dt, and consequently ˆf (ω) = ω ˆf(ω). Therefore (log ˆf(ω)) = ω = ˆf(ω) = ˆf() e ω2 /2. Since ˆf() =, we obtain ˆf(ω) = e ω2 /2 as before. We will see later that the Heisenberg Uncertainty Principle is a consequence of this important example. 7
3 3 2 2.5 4 2 2 4.5 4 2 2 4 f(t) ˆf(ω) Figure 3: Fourier transform pair for the Gaussian bell function C! # " L! Figure 4: Contour of integration +L 8