8. Convolution. T is linear: T (af + bg) = at (f) + bt (g) (8.1) T is translation invariant: T (τ a (f)) = τ a (T (f)) (8.2)

Transcription

1 8. Convolution 8.. Linear Translation Invariant (LTI or LSI) Operators An extremely important class of operators is the class of linear translation invariant or linear shift invariant operators. These are operators T that satisfy the following three conditions: (i) (ii) (iii) T is bounded or continuous on some space somehow. T is linear: T (af + bg) = at (f) + bt (g) (8.) T is translation invariant: T (τ a (f)) = τ a (T (f)) (8.2) The model that we want to think of is that T is a black box a processor into which we feed a function, or signal, f. The output is the processed signal T (f). We ll be very loose about what we mean by condition (i); intuitively it means that a small change in the input signal should result in a small change in the output. We are being deliberately vague for now about what we mean by small. Another physical name for (ii) is that our operator or processor obeys a principle of superposition. The intuitive meaning of property (iii) is this: the system respond in the same way to the same input, regardless of what the clock on the wall says. If we feed a certain song into the amplifier, we get a certain response from the speakers. If we feed the same song into the amplifier, only three hours later, we get the same response from the speakers three hours later. Since the concept is so basic, let us recognize for the purpose of this course the standard abbreviation LTI operator. Some examples of LTI operators: the zero operator, the identity operator, any translation operator τ a ; multiplication by a constant, differentiation; any linear constant coefficient differential operator; the inverse of a linear constant coefficient differential operator, the partial sum operators S M defined in (5.2). We ll have some others to consider, called filters, that do various things to various frequencies of the input signal; the partial sum operators are examples of low-pass filters. Does the Fourier transform interact with LTI operators in a predictable way? The answer is a resounding yes. What follows is going to be an extremely formal argument. To put even the barest hint of rigor into it requires either the language of generalized functions or a whole lot of functional analysis. Suppose that we have an LTI operator T, and suppose that the set of functions T is capable of acting on includes all of the simple harmonic motions. Just for this argument, let us adopt a temporary notation: for each frequency s, the function e s is defined as e s (x) = e 2πisx. An important identity is the one we obtain by letting a translation act on e s. τ a e s (x) = e s (x + a) = e 2πis(x+a) = e 2πisa e 2πisx = e 2πisa e s (x) (8.3) How does our LTI operator act on a simple harmonic motion? We test it by applying translations. T e s (x + a) = τ a T e s (x) = T τ a e s (x) = T ( e 2πisa e s (x) ) = e 2πisa T e s (x) (8.4) Let x = in equation (8.4): T e s (a) = e 2πisa T e s () (8.5) 57

2 Now replace a in (8.5) by x : T e s (x) = e 2πisx T e s () = T e s ()e s (x) (8.6) We have discovered that an LTI operator sends a simple harmonic motion to a constant multiple of itself. What about arbitrary functions f(x)? We use the synthesis equation: f(x) = f(s)e s (x) ds (8.7) Hit both sides of (8.7) with the operator T. Use the analogy of sums to integrals and the linearity of the operator to bring T inside the integral. (Yes, this is a very formal derivation.) T f(x) = f(s)t e s (x) ds = f(s)t e s ()e s (x) ds (8.8) T e s () is a number that depends only on s. Let s give it a new name: T e s () = m T (s). T f(x) = f(s)m T (s)e 2πisx ds (8.9) If we read (8.9) as being another synthesis equation, then we can extract the following from equation (8.9). T f(s) = m T (s) f(s) (8.) A paraphrase of equation (8.) goes like this: every LTI operator is given by point-by-point multiplication on the Fourier transform side. The function m T (s) is called the multiplier associated with the operator T. (Engineers might call m T (s) the transfer function.) We wrote (8.) as if we were talking about R; however, the same identity is true on each of the other three domains. An LTI opererator acting on functions performs the same action as multiplying the Fourier transforms by some fixed function. For a different perspective on what this means, consider LTI operators on functions on P N. A function on P N is an N-vector and the set of all such functions is an N-dimensional vector space. Any linear operator (linear transformation) from this vector space into itself is given by the multiplication of the N-vector by an N N matrix. The set of LTI operators is then some (possibly fairly small) proper subset of the set of all N N matrices. (In fact, these matrices have the special property that they are constant on all wrapped-around diagonals in the same direction as the main diagonal, and they have a special name: circulant matrices.) Let T be some LTI operator. Then there is a function m T [k] on P N such that for all x functions on P N } : T x[k] = m T [k] x[k] (8.) Now let s look at this from a matrix point of view. Consider both x and x to be N column vectors, and let the N N matrix of the operator T also called T. Let F be the matrix whose (j, k)th element is ζ jk = e 2πijk/N. Let F be the conjugate transpose of F, which is to say the matrix whose (j, k)th element is ζ jk = e 2πijk/N. (This is the same matrix defined in equation (4.).) The work in chapter 4 shows that N F F = I, where I is the identity matrix. (8.2) 58

3 The analysis equation is x = N F x, and the synthesis equation is x = F x. Equation (8.) can be rewritten as N F Tx equals a vector obtained by multiplying the kth component of the vector N F x by m T [k]. But the act of each component of a vector by a different number can be written as multiplication of that vector by a diagonal matrix. Let D = D T be the diagonal matrix obtained by making the kth entry on the diagonal m T [k] and making all other entries of that matrix zero. Then we have the following further rewrite of (8.): ( ) N F Tx = D N F x Multiply both sides of this on the left by F. ( ) ( ) Tx = FD N F x or T = FD N F or N F TF = D (8.3) To put a name to what we have here, the Fourier transform diagonalizes all LTI operators. The numbers m T [k] are the eigenvalues of T Introduction to convolution From looking at LTI operators, we develop an interest in the following: Suppose we have functions f(x) and g(x). We calculate their Fourier transforms f(s) and ĝ(s), multiply these two transforms together and assume that the result is the Fourier transform ĥ(s) of some function h(x), then use Fourier inversion (the synthesis integral) to recover h. That is, Then, h(x) = = = = ĥ(s) = f(s)ĝ(s); ĥ(s)e 2πisx ds = f(s)ĝ(s)e 2πisx ds [ ] f(s) g(u)e 2πisu du e 2πisx ds [ ] g(u) f(s)e 2πis(x u) ds du f(x u)g(u)du (8.4) The (binary) operation that combines two functions given in (8.4) is called the convolution of the two functions. Let s define what we mean by convolution in each of our four domains: Convolution on R : Convolution on Z : Convolution: f g(x) = f(x u)g(u) du (8.5) Fourier transform: f g (s) = f(s)ĝ(s) (8.6) Convolution: f g[n] = 59 m= f[n m]g[m] (8.7)

4 Convolution on T P : Fourier transform: f g [k] = q f [k] ĝ [k] (8.8) Convolution: f g(x) = P P f(mod(x u, p))g(u) du (8.9) Fourier transform: f g [k] = f [k] ĝ [k] (8.2) Warning! We had a notational choice: should we include the in equation (8.9) or not? P Leaving it out would be consistent with all of the other equations in this section, and indeed that is the choice that Kammler would make, but I have another agenda here which argues in favor of the notation I chose. We ll just have to watch our step. Convolution on P N : Convolution: f g[n] = N m= f[mod(n m, N)]g[m] (8.2) Fourier transform: f g[k] = N f[k]ĝ[k] (8.22) Convolution on one side yields multiplication on the other side. Many things in this subject are symmetrical, so multiplication on one side should yield convolution on the other side. Without further ado, let s write down the formulas: On R : If h(x) = f(x)g(x), then ĥ(s) = ( f ĝ)(s) (8.23) On T P : If h(x) = f(x)g(x), then ĥ[k] = ( f ĝ)[k] (8.24) On Z : If h[n] = f[n]g[n], then ĥ(s) = q( f ĝ)(s) = q f(s u)ĝ(u) du (8.25) On P N : If h[n] = f[n]g[n], then ĥ[k] = ( f ĝ)[k] (8.26) Our funny notational choice in (8.9) came back to haunt us on (8.25), introducing what looks like an extraneous multiplication by q into the formula. We ll just have to live with it Convolution as a multiplication Convolution is a binary operation on functions. It takes as input two functions and delivers as output another function. As a binary operation, it has many of the characteristics of a multiplication. Let s look at what some of those properties are. Convolution is commutative. To see this, write out the integral in (8.5), then make the change of variable y = x u, from which u = x y. By Calculus I notation, du = dy, but the domain of integration gets flipped around, balancing out the minus sign. If we follow the discussion before equation (7.3), we will just write du = dy, with the same result. f g(x) = f(x u)g(u) du = f(y)g(x y) dy = g f(x) (8.27) 6

5 Very similar arguments could be made in each of the other three domains. Convolution is associative. This means that f (g h) = (f g) h. The proof of this identity is left to the reader. It s a lot of work, but the work is straightforward. You do what you have to do. Convolution is bilinear. This means that f (ag + bh) = af g + bf h and (af + bg) h = af h + bg h. Eventually, some of this is going to lead to the topic of commutative Banach algebras. There is one multiplication-related point I m still hiding from you: I haven t told you whether there is or isn t an identity element for convolution A practical guide for convolving piecewise-defined functions The trickiest part of dealing with convolution is figuring out what to do when one or both of the functions being convolved is piecewise defined: e.g., if x is less than a certain number, f(x) is defined by a certain formula, if x is greater or equal to that number but less than another number, f(x) is defined by another formula, and so forth. The following steps might help.. Don t forget that convolution is commutative - that is, (g f)(x). Try writing the integral in (8.5) both ways, so that you can pick the way that looks easier.. Inspect the definitions of both f(x) and g(x) to determine for each function the best interval that describes where the function is anything but zero: outside this interval the function is x if x < 2 zero. For instance, if f(x) = x if 3 x < 4 then that interval is [, 4]. (We include the otherwise endpoints as a matter of routine and convention: it s frequently not particularly important to us exactly what the definition of the function is at any one point.) The interval could easily be of the form [a, ) or (, b]. You have two such intervals for the two functions; call them [a, b] and [c, d]. Then the convolution of the two functions will live on the interval [a + c, b + d] in the sense that (f g)(x) will be for x < a + c and for x > b + d, and we don t need to do anything else about computing (f g)(x) out there. 2. For each of f(x) and g(x) write out the list of the change points all of the points at which the formula that defines the function changes. For instance, the function f(x) listed in step has as its change points, 2, 3, 4}. If either function has any absolute values symbols in its formulas, list all of the places at which the thing inside the absolute value equals zero as change points. For instance, if g(x) = x 2 4, then the change points of g are 2, 2} and if x if x < /\(x) =, then the change points of /\ are,, }. otherwise 3. You now have two lists the change points of f and the change points of g. Find the list of all of the numbers that you can obtain by adding together any element of the first list with any element of the second list. The resulting list is the list of the change points of (f g)(x). For example, suppose the change points of f(x) are, 2, 3, 4} and the change points of g(x) are,, }. Then the change points of (f g)(x) are,,, 2, 3, 4, 5}. With 3 points on 6

6 one list and 4 points on the other, there could have been as many as 3 4 = 2 points on the list of sums, but in this case some of these sums were duplicated. (Actually, a function with 7 change points is one that could take 8 lines to describe we hope that most of our problems aren t quite that complicated!) 4. Proceed on a case-by-case basis. For each interval between adjacent change points of (f g)(x) (or outside, if outside matters ), assume that x belongs to that interval. Write out the integral: f(x u)g(u) du Break this integral down into as many integrals over separate intervals as there are non-zero pieces of the definition of g(x). For each such separate integral, determine which parts of the definition of f(x) apply within that interval. For that purpose, you need to know the answers to questions of this form: for what values of u is x u in the interval [a, b]? The answer is for x b u x a. This allows you to subdivide your integral even further into integrals whose lower limit of integration may be of the form x b or whose upper limit of integration may be of the form x a. Add up these pieces of integrals to get the value of (f g)(x) on the particular interval for x that you are working on; repeat for each of the other intervals. 5. Sketch the graphs of f(x), g(x), and (f g)(x). Expect that (f g)(x) will be the smoothest of the bunch; in particular, expect that under ordinary circumstances, (f g)(x) will be continuous even if f(x) and g(x) are not. Let s illustrate this with two examples: if x 3 if x 2 Example 8.4a: Let f(x) = and g(x) =. otherwise otherwise By step, we see that f(x) is non-zero on [, 3] and g(x) is non-zero on [, 2] so that (f g)(x) should be nonzero on (at most) [, 3] + [, 2] = [2, 5]. By step 2, we see that our lists of change points are, 3} and, 2} so that by step 3 the list of change points of (f g)(x) is 2, 3, 4, 5}. We have already decided that for values of x that are outside which is to say less than 2 or greater than 5 then (f g)(x) is zero. This leaves us three cases for values of x. First case: Assume 2 x 3. (We re going to be sloppy about whether these should be < or ; it really won t matter to the integral calculations that follow.) We must compute f(x u)g(u) du = 2 f(x u) du But f(x u) is equal to if x u in the interval [, 3], which is to say when x 3 u x, and is zero otherwise. Since x is between 2 and 3, x 3 is between and and thus doesn t matter when we are integrating from to 2, but x is between and 2 and matters very much. f is to the left of that point and otherwise, so we have x du = (x ) = x 2. Second case: Assume 3 < x < 4. Now x 3 is between and and x is between 2 and 3: neither one of these change points matters when we are integrating from to 2. In fact, f(x u) 62

7 is equal to on the whole of that interval and we have 2 du = 2 =. Third case: Assume 4 < x < 5. Now x 3 is between and 2 and matters: f is equal to for values of u to the right of x 3. x is between 3 and 4 and does not matter. 2 x 3 du = 2 (x 3) = 5 x. Overall, we have found that x 2 if 2 x 3 if 3 x 4 5 x if 4 x 5 otherwise The graph looks like this (note that (f g)(x) is continuous, even though neither f(x) nor g(x) is): f g(x) Example 8.4b: Let f(x) = e x if x if x > and g(x) = e x if x if x <. The interval on which (f g)(x) can be non-zero is (, ] + [, ) = (, ), so we have to consider all possible values of x. The only change point of f is and the only change point of g is also, so the only change point of (f g)(x) is and we have two cases to consider. In any case, we must compute this integral: f(x u)g(u) du = f(x u)e u du First case: Assume that x <. Then f(x u) = e x u if x ule, which is to say if u x. Since x <, this holds for all of the interval [, ) upon which we are integrating. e x u e u du = infty e x e 2u du = 2 ex e 2u = 2 ex 63

8 Second case: Assume that x. Then f(x u) = e x u if u x, but is zero otherwise. Since x lies inside the interval [, ), this matters. x e x u e u du = x e x e 2u du = 2 ex e 2u = 2 ex e 2x = 2 e x In this case, we can combine the two formulas into one line as 2 e x. Once again, this is a continuous function The Wave Equation on the Line We consider the one-dimensional wave equation for the infinite line: 2 u t 2 = c2 2 u u for < x <, < t < ; u(x, ) = f(x), (x, ) = g(x). (8.28) x2 t Here c is a physical constant that has the dimensions of a velocity. We ll see that c is the speed T of wave propagation. For a string under tension T with mass per unit length ρ, c = ρ. We take the Fourier transform of the equation with respect to x, the space variablel arriving at 2 t 2 (û) = 4π2 c 2 s 2 û; û (s, ) = f(s); x ) = ĝ(s) (8.29) tû(s, This is now an ordinary differential equation in t. In particular, it is the ODE of simple harmonic motion of angular frequency 2πcs, that is of frequency cs. The solution is û(s, t) = A(s) cos(2πcst) + B(s) sin(2πcst) (8.3) Considering the initial conditions in (8.29) leads us to saying that A(s) = f(s) and that B(s) = ĝ(s) 2πcs. We now write our answer as u(x, t) = u (x, t) + u 2 (x, t) where û (s, t) = f(s) cos(2πcst) and û 2 (s, t) = ĝ(s) sin(2πcst) 2πcs We deal with these terms one at a time. û (s, t) = f(s) cos(2πcst) = 2 f(s)e 2πicst + 2 By the translation formula (7.3), we get that f(s)e 2πicst (8.3) u (x, t) = 2 f(x + ct) + f(x ct) (8.32) 2 Formula (8.32) is quite striking. It is the classic solution due to DAlambert in which the (stationary) initial position is divided in two with half of it sent sliding to the right at speed c without changing shape and the other half sliding to the left at the same speed. Now we deal with the part that depends on the initial velocity. By the second part of (8.3), û 2 is the product of ĝ and another function of s, which by (8.6) makes u 2 the result of the convolution of g with the function whose Fourier transform is sin(2πcst) = t sinc(2cts). We know that sinc(s) 2πcs 64

9 is the Fourier transform of (x). The dilation rule (7.3) tells us that the function whose Fourier t ( x ) transform is t sinc(2cts), is 2ct = ( x ) 2ct 2c. Explicitly writing out the convolution, we 2ct have u 2 (x, t) = 2c x+ct x ct g(y) dy (8.33) Our final answer for u is simply the sum of the functions in (8.32) and (8.33) Exercises 8.. On R, define the operator T f(x) = ( + x 2) f(x). (a) Show that T is a linear operator. (b) Show by appropriate example that T is not translation invariant Using the functions defined in Example 8.4a, compute f(s), ĝ(s) and f g(s). Verify that identity (8.6) is true in this case Using the functions defined in Example 8.4b, compute f(s), ĝ(s) and f g(s). Verify that identity (8.6) is true in this case Consider the wave equation on the whole line with c = : conditions being that u(x, ) = (zero intial position) and u for x (x, ) = t otherwise. 2 u t 2 = 2 u, with the initial x2 (This simulates a string being struck a sudden blow by a broad, blunt hammer.) Find the solution u(x, t) and draw sketches of the graph for t = 2,, 3, 2, 3, and