1.3 Frequency Analysis A Review of Complex Numbers

Transcription

1 3 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS I y z R θ x R Figure.8. A complex number z can be represented in Cartesian coordinates x, y or polar coordinates R, θ..3 Frequency Analysis.3. A Review of Complex Numbers A complex number is represented in the form z x + jy, where x and y are real numbers satisfying the usual rules of addition and multiplication, and the symbol j, called the imaginary unit, has the property j. The numbers x and y are called the real and imaginary part of z, respectively, and are denoted by x Rz, y Iz. We say that z is real if y, while it is purely imaginary if x. Example.. The complex number z 3+j has real part 3 and imaginary part, while the real number 5 can be viewed as the complex number z 5+j whose real part is 5 and imaginary part is. Geometrically, complex numbers can be represented as vectors in the plane Fig..8. We will call the xy-plane, when viewed in this manner, the complex plane, with the x- axis designated as the real axis, and the y-axis as the imaginary axis. We designate the complex number zero as the origin. Thus, x + jy means x y. In addition, since two points in the plane are the same if and only if both their x- and y-coordinates agree, we can define equality of two complex numbers as follows: x + jy x + jy means x x and y y. Thus, we see that a single statement about the equality of two complex quantities actually contains two real equations.

2 Sec..3. Frequency Analysis 33 Definition. Complex Arithmetic. Let z x + jy and z x + jy. Then we define: a z ± z x ± x +jy ± y ; b z z x x y y +jx y + x y ; c for z, w z z is the complex number for which z z w. Note that, instead of the Cartesian coordinates x and y, we could use polar coordinates to represent points in the plane. The polar coordinates are radial distance R and angle θ, as illustrated in Fig..8. The relationship between the two sets of coordinates is: x R cos θ, y R sin θ, R x + y z, y θ arctan. x Note that R is called the modulus, or the absolute value of z, and it alternatively denoted z. Thus, the polar representation is: z z cos θ + j z sin θ z cos θ + j sin θ. Definition. Complex Exponential Function. The complex exponential function, denoted by e z,orexpz, is defined by e z e x+jy e x cos y + j sin y. In particular, if x, we have Euler s equation: e jy cos y + j sin y. Comparing this with the terms in the polar representation of a complex variable, we see that any complex variable can be written as: z z e jθ. Properties of Complex Exponentials. cos θ ejθ + e jθ, sin θ j ejθ e jθ, e jθ, e z e z e z +z, e z e z, e z+πjn e x cosy +πn+jsiny +πn e x cos y + j sin y e z, for any integer n.

3 34 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS z z z z z z I θ θ + θ y z θ θ x R Figure.9. Multiplication of two complex numbers z z e jθ and z z e jθ z is not shown. The result is z z e jθ with z z z and θ θ + θ. DT complex exponential functions whose frequencies differ by π are thus identical: e jω+πn e jωn+πjn e jωn. We have seen examples of this phenomenon before, when we discussed DT sinusoids. It follows from the multiplication rule that z z z e jθ z e jθ z z e jθ +θ. Therefore, in order to multiply two complex numbers, add the angles; multiply the absolute values. Multiplication of two complex numbers is illustrated in Fig..9. Definition.3 Complex Conjugate. If z x + jy, then the complex conjugate of z is z x jy sometimes also denoted z. This definition is illustrated in Fig..a. Note that, if z z e jθ, then z z e jθ. Here are some other useful identities involving complex conjugates: Rz z + z, Iz j z z, z zz, z z, z z z is real, z + z z + z, z z zz.

4 Sec..3. Frequency Analysis 35 I I z j y z x + jy x R z +j, z θ π/4 R /z / j/ y z x jy z j a b Figure.. a Complex number z and its complex conjugate z. b Illustrations to Example.. Example.. Let us compute the various quantities defined above for z +j.. z j.. z +. Alternatively, z zz + j j +j j j. 3. Rz Iz. 4. Polar representation: z cos π 4 + j sin π 4 e j π To compute z, square the absolute value and double the angle: z cos π + j sin π j ej π. The same answer is obtained from the Cartesian representation: + j + j +j + j +j j. 6. To compute /z, multiply both the numerator and the denominator by z : z z zz j + j j j Alternatively, use the polar representation: z e j π 4 e j π 4 j. cos π + j sin π 4 4 j j.

5 36 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS We can check to make sure that /z z : These computations are illustrated in Fig..b..3. A Review of Basic Linear Algebra Frequency analysis involves studying representations of the form j +j j + j j. s k a k g k,.6 where signal s which is being analyzed, is written as a linear combination of a set of orthogonal sinusoidal signals g k. For example, Eq..6 is called DT Fourier series if signals g k are DT complex exponential signals representing different frequency components of a periodic DT signal s. Similarly, if g k are CT complex exponential signals and s is a CT periodic signal, representation.6 becomes CT Fourier series. There are, moreover, other ways of decomposing a signal into its basic components which are not necessarily frequency components. As a matter of fact, we have already seen a decomposition like this when deriving the convolution formula Section..3 where g k δ k were shifted unit impulse signals. There are many other reasons some of which will become clear later in the course for studying representations of the form.6 in general, rather than focusing solely on Fourier series. We pose the following questions regarding Eq..6:. Given an arbitrary signal s and a set of pairwise orthogonal signals g k, does representation.6 exist? In other words, can we find such coefficients a k that Eq..6 is satisfied?. If so, what are the coefficients a k? 3. If not, can we at least find coefficients a k such that Eq..6 is approximately satisfied? Precise answers to these three questions are impossible unless we define what we mean by orthogonal and approximately. In order to do this, we generalize the notions of orthogonality and length found in planar geometry, to spaces of signals, via the following procedure: define an appropriate space of signals; define what it means for two signals to be orthogonal, by appropriately generalizing the notion of orthogonality of two vectors in a plane; define the distance between any two signals, by appropriately generalizing the notion of distance in a plane.

6 Sec..3. Frequency Analysis 37 We address the first item on this agenda by initially restricting our attention to complexvalued DT signals defined on a fixed interval, say, [,N ] where N is some fixed nonnegative integer. In other words, we consider DT signals whose domain is the set {,,...,N } and whose range is C. Each such signal s can be represented as an N-dimensional vector s, by recording its N samples in a column: s s s. sn The collection of all such signals is therefore the same as the set of all N-dimensional complex-valued vectors which we call C N. Several remarks about notational conventions are in order here. In writing, vectors are usually denoted by underlining them: s. In printed texts, however, it is customary to use boldface letters s to denote vectors. A transpose of a column vector is a row vector that is, an equivalent expression for s is: s s,s,...,sn T. We will occasionally be using vectors to represent signals defined for n,,...,n rather than n,,...,n. In this case, s s s.. sn Although we will mostly work with complex-valued signals, sometimes it is useful to consider only real-valued signals. The corresponding set of all N-dimensional real-valued vectors is called R N. Since any real-valued vector can be viewed as a complex-valued vector with a zero imaginary part, R N is a subset of C N. Even though the domain of definition of our signals is a set consisting of N points, it is sometimes helpful to pretend that the signals are actually defined for all integer n. Two most commonly used ways of doing this are padding with zeros and periodic extension. The former assumes that the signal values outside of n,,...,n are all zero:. sn, n < orn N. The latter assumes that the signals under consideration are periodic with period N: sn sn mod N, n < orn N, in other words, sn s, sn +s, etc.

7 38 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS x y A y C y B D O x y x x y x Figure.. Two vectors in the real plane are orthogonal if and only if they form a 9 angle. We will often be using symbol which means is an element of. For example, s C N means: s is an element of C N. Inner Products and Orthogonality Definition.4. The inner product of two vectors s C N and g C N is denoted by s, g and is defined by s, g N n sngn. Two vectors s and g are defined to be orthogonal denoted s g if their inner product is zero: s g means s, g. Example.. Let us see whether our definition of orthogonality makes sense for vectors in the real plane R. Two vectors in the plane are called orthogonal if the angle between them is 9 x.lets R x and g R be two vectors in the y real plane shown in Fig... Their inner product is then s, g x x + y y. If the inner product is equal zero, then x y, y x which says that the right triangles OBA and CDO are similar. From the similarity of these triangles, we get AOB OCD. Therefore, y COA 8 AOB DOC 8 OCD DOC 9. Similar reasoning applies if the two vectors are oriented differently with respect to the coordinate axes. Therefore, saying that the inner product of two vectors in the real

8 Sec..3. Frequency Analysis 39 plane is zero is equivalent to saying that they form a 9 angle. This shows that our definition of orthogonality in C N is an appropriate generalization of orthogonality in the real plane. It is easily seen that the inner product of a vector with itself is always a real number: s, s N n snsn This number is called the energy of the vector. N n sn. Definition.5. The inner product of a vector s with itself is called the energy of s. The square root of the energy is called the norm 4 of s, and is denoted s : s s, s. Example.3. For example, the norm of the vector x y in Fig.. is x + y which is simply the length of the segment OA. Our definition of a norm therefore generalizes the familiar concept of the length of a vector in the real plane. Here are some properties of inner products and norms:. g, s s, g.. a s + a s, g a s, g + a s, g. 3. s,a g + a g a s, g + a s, g. 4. as a s. 5. Pythagoras s theorem: the sum of energies of two orthogonal vectors is equal to the energy of their sum, i.e., if s, g, then s + g s + g. Fig.. and the two examples above illustrate the fact that our definitions of orthogonality and the norm in C N generalize the corresponding concepts from planar geometry. Therefore, when considering vectors in C N it is often helpful to draw planar pictures to guide our intuition. Note, however, that the proof of any facts concerning inner products and norms in C N for example, the properties listed above cannot be based solely on pictures: the pictures are there to guide our reasoning, but rigorous proofs must rely only on definitions and properties proved before. For example, in proving Property above we can only rely on our definition of the inner product. Once Property is proved, we can use both Property if we need to and the definition of the inner product in proving Property, etc. 4 We will soon see that the term norm is actually more general. The specific norm that is the square root of the energy is called the Euclidean norm or the l norm or the -norm.

9 4 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS s s g ag s s g g s s g g sg Projection of s onto G: s G s g + s g g Space G, the span of g and g. a b Figure.. a The orthogonal projection s g of a vector s onto another vector g. b The orthogonal projection s G of a vector s onto a space G can be obtained by projecting s onto g and g and adding the results, where {g, g } is an orthogonal basis for G. Orthogonal Projections In the real plane, the coordinates of a vector are given by the projections of the vector x onto the coordinate axes. For example, the projections of the vector in Fig.. y x onto the x-axis and y-axis are the vectors and, respectively. We will y soon see that the coordinates of a signal in a Fourier basis that is, the Fourier series coefficients can also be computed from the projections of the signal onto the individual Fourier basis functions. We define the projection of a vector s C N onto another vector g C N by generalizing the notion of an orthogonal projection from planar geometry. Specifically, when we project s onto g, we get another vector s g which is collinear with g i.e. it is of the form ag such that the difference s s g is orthogonal to g. An illustration of this, for the real plane, is given in Fig..a. Definition.6. The orthogonal projection of a vector s C N onto a nonzero vector g C N is such a vector s g C N that:. s g ag for some complex number a, and. s s g g. The orthogonal projection of a real-valued vector s R N onto another real-valued vector g R N is defined similarly. Note again that, even though our definition applies to the general case of C N,we can use the planar picture of Fig..a to guide our analysis. From this picture, we

10 Sec..3. Frequency Analysis 4 y x Figure.3. Illustration to Example.4. immediately see that the coefficient a in the expression s g ag is not arbitrary. If it is too small or too large, the angle between s s g and g will not be 9. To find the correct coefficient a, note that the two conditions in the definition above imply that s ag, g. But this equation can be rewritten as follows: s, g a g, g. Therefore, a s, g g, g s g ag s, g g. g, g.7 Example.4. Does our result of Eq..7 make sense for -dimensional real vectors? Suppose that we want to project the vector s onto the vector g. As shown in Fig..3, the result should clearly be s g give the same answer? s g,, Does our formula.7

11 4 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS Our definition works as expected in the plane. Just as we did with the concepts of orthogonality and length before, we have generalized the planar concept of orthogonal projection to C N. We now generalize our orthogonal projection formula.7 to the case when we project a vector s onto a space G. This is illustrated in Fig..b. We show that, if an orthogonal basis for space G is known, it is easy to compute the projection of any vector s onto G. Of course we need to precisely define what we mean by an orthogonal basis before we proceed. Definition.7. A subset G of C N is called a vector subspace of C N if. ag G for any g G and any a C,. and g + g G for any g, g G. A subset G of R N is called a vector subspace of R N if. ag G for any g G and any a R,. and g + g G for any g, g G. In other words, a set G in C N or in R N is called a vector subspace if it is closed under multiplication by a scalar and under vector addition. α Example.5. The set of all vectors of the form where α C, is a vector subspace of C 3 : if you multiply a vector of this form by a complex number, you get a vector of this form; if you add two vectors of this form, you again get a vector of this form. On the other hand, this set of vectors is not a vector subspace of R 3 simply because it is not a subset of R 3. α The set of all vectors of the form where α C, is not a vector subspace of α C 3 : if you multiply a vector like that by, you get which is no longer in the set. Definition.8. Vectors g, g,...,g m are called linearly independent if none of them can be expressed as a linear combination of the others: g i k i a k g k for i,,...,m. Equivalently, g, g,...,g m are called linearly independent if a g + a g a m g m implies a a...a m.

12 Sec..3. Frequency Analysis 43 Definition.9. The space spanned by vectors g, g,...,g m is the set of all their linear combinations, i.e. the set of all vectors of the form a g + a g a m g m, where a,a,...,a m are numbers complex numbers if we are in C N, real numbers if we are in R N. Definition.. If G span{g,...,g m } and if g,...,g m are linearly independent, then {g,...,g m } is said to be a basis for space G. If, in addition, g,...,g m are pairwise orthogonal, the basis is said to be an orthogonal basis. { } x Example.6. span, C since x + y for y x any C y. It is an easy exercise to show that and are linearly independent and, moreover, orthogonal. Therefore, these two vectors form an orthogonal basis for C. We will need the following important result from linear algebra which we state here without proof. Theorem.. Any N linearly independent vectors in C N R N form a basis for C N R N. Any N pairwise orthogonal nonzero vectors in C N R N form an orthogonal basis for C N R N. We now define the orthogonal projection s G of a vector s onto a subspace G of C N. We use the 3-dimensional picture of Fig..b as a guide. First, s G must lie in G. Second, the difference between s and s G must be orthogonal to G. Definition.. The orthogonal projection of a vector s C N onto a subspace G of C N is such a vector s G C N that:. s G G, and. s s G G, that is, s s G is orthogonal to any vector in G. The orthogonal projection of a real-valued vector s R N onto a subspace G of R N is defined similarly. Let s G be the orthogonal projection of s onto a subspace G of C N. Suppose that {g,...,g m } is an orthogonal basis for G. One consequence of this is that any vector in G can be represented as a linear combination of vectors g,...,g m. In particular, since s G G, we have that m s G a k g k for some complex numbers a,...,a m..8 k

13 44 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS These coefficients a,...,a m cannot be any arbitrary set of numbers, however: they have to be such numbers that s s G is orthogonal to any vector in G. In particular, s s G g, s s G g,..., s s G g m : s s G, g p, for p,...,m s, g p s G, g p m s, g p a k g k, g p s, g p k m a k g k, g p. k But notice that the orthogonality of the basis {g,...,g m } implies that g k, g p unless p k. Therefore, only one term in the summation can be nonzero the term for k p: s, g p a p g p, g p a p s, g p g p, g p for p,...,m. Note that, since {g,...,g m } is a basis, g p and therefore g p, g p. This means that dividing by g p, g p is allowed. Substituting the expression we obtained for the coefficients into Eq..8, we obtain: s G m k s, g k g k, g k g k..9 Comparing this result with our result.7 for projecting one vector onto another, we see that projecting onto a space G which has an orthogonal basis {g,...,g m } amounts to the following: project onto the individual basis vectors; sum the results. This is illustrated in Fig..b for projecting a 3-dimensional vector onto a plane spanned by two orthogonal vectors g and g. Formula.9 is actually a lot more than a formula for projecting a vector onto a subspace. Note that, if the vector s belongs to G then its projection onto G is equal to the vector itself: s G s if s G. In this case, Eq..9 tells us how to represent s as a linear combination of orthogonal basis vectors.

14 Sec..3. Frequency Analysis 45 s s f s s G s G f s G, the orthogonal projection of s onto space G. f Space G with orthogonal basis {g,...,g m }. Figure.4. Illustration to the proof of Theorem.4: The closest point in space G to a fixed vector s is the orthogonal projection s G of s onto G. Theorem.3 Decomposition and Reconstruction in an Orthogonal Basis. Suppose that {g,...,g m } is an orthogonal basis for a subspace G of C N in particular G could be C N itself, and suppose that s G. Then s m a k g k, synthesis or reconstruction formula. k where a k s, g k g k, g k for k,...,m. analysis or decomposition formula. The coefficients. are unique, i.e. there is no other set of coefficients that satisfy Eq... As we will shortly see, a particular case of these equations are the DT Fourier series formulas. These equations, however, are very general: they work for non-fourier bases of C N. Slight modifications of these equations also apply to other spaces of DT and CT signals. For example, the CT Fourier series formulas are essentially a particular case of Eqs..,.. Suppose now that s does not belong to the subspace G. Can we find coefficients a,...,a m such that Eq.. is satisfied approximately? Specifically, we would like to find the coefficients that minimize the energy or, equivalently, the -norm of the

15 46 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS error i.e. the -norm of the difference between the two sides of.: m find a,...,a m to minimize s a k g k. It turns out that the answer is still Eq.. i.e. that the orthogonal projection s G of s onto G is the closest vector to s among all vectors in G. To show this, consider Fig..4. Using the definition of orthogonal projection, we see that the vector s s G must be orthogonal to G, i.e. k s s G v for any v G. For any arbitrary vector f G, we have s G f G. Hence s s G s G f. We can therefore apply Pythagoras s theorem to the triangle formed by vectors s f, s s G, and s G f: s f s s G +s G f s s G + s G f s s G. Therefore, s s G s f for any f G, or alternatively, s s G min s f. f G So s G is the closest vector in G to s. It is easily seen, moreover, that equality is achieved only if f s G which means that s G is the unique closest vector to s. Theorem.4 Approximation by an Orthogonal Set of Vectors. Suppose that {g,...,g m } is an orthogonal basis for a subspace G of C N in particular G could be C N itself, and suppose that s C N is a vector which may or may not belong to G. We seek to approximate s by a vector in G. If we look for an approximation ŝ G which minimizes the -norm ŝ s of the error, then ŝ m a k g k,. k where a k s, g k g k, g k for k,...,m..3 The coefficients.3 are unique, i.e. there is no other set of coefficients that results in the minimum -norm of the error.

16 Sec..3. Frequency Analysis Discrete-Time Fourier Series and DFT Example.7. Let g and g be the following two discrete-time complex exponential functions defined for n,, 3, 4: jπn g n exp, n,, 3, 4; 4 jπn and g n exp, n,, 3, 4. 4 a Suppose that, n +j, n sn, n 3 j, n 4. Can the signal s be represented as a linear combination of g and g? If so, find coefficients a,a in this representation: sn a g n+a g n, n,, 3, 4. b Suppose that, n s, n n, n 3, n 4. Can the signal s be represented as a linear combination of g and g? coefficients a,a in this representation: If so, find s n a g n+a g n, n,, 3, 4. Solution. a Write all three signals as vectors, i.e., g g g g 3, g g 4 g g g 3 g 4, s What are the entries of these vectors? jπ g exp expj, 4 jπ g exp exp j π, 4 s s s3 s4.

17 48 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS I g j g 3 R g g 4 j Figure.5. plane C. Illustration to Example.7: The four entries of vector g as points in the complex jπ g 3 exp expjπ, 4 jπ3 g 4 exp exp j 3π 4 Fig..5 shows a plot of these in the complex plane recall that expjθ has absolute value and angle θ. Calculations for g are similar. We obtain: g j j, g. One method of solving this problem would be to simply notice that s +j g + g by inspection. j We have thus represented s as a linear combination of g and g, with coefficients a a : sn g n+g n n,, 3, 4. Another method is to notice that g and g are orthogonal, g, g + j + + j j +j..

18 Sec..3. Frequency Analysis 49 s g 4 Projection of s onto G: g 4 + g g 4 4 g s g + g g Space G, the span of g and g. Figure.6. Illustration to Example.7: Vector s lies in the space spanned by g and g, and therefore can be represented as their linear combination. Vector s is not in the space spanned by g and g and cannot be represented as a linear combination of g and g. The closest linear combination of g and g to s is the orthogonal projection of s onto the span of g and g. We can therefore use Theorem.3: if s is representable as a linear combination of g and g then the coefficients are found from: a s, g g, g + +j j + + j j + j j + + j j +j +++ j +, +++ a s, g g, g + +j + + j j+++j. +++ Since g +g s, the answer is that s can indeed be represented as a linear combination of g and g, with coefficients a a. Geometrically, this means that s lies in the space spanned by g and g, as illustrated in Fig..6. b We take the same approach as in Part a: if s is representable as a linear

19 5 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS combination of g and g then the coefficients are found from: However, a s, g g, g + j + + j 4 a s, g g, g g + 4 g j 4 / j 4 s. 4, 4. Therefore, s cannot be represented as a linear combination of g and g. Geometrically, this means that s lies outside of the space spanned by g and g, as illustrated in Fig..6. The coefficients a /4 and a /4 we computed are actually the coefficients of the orthogonal projection of s onto this space. Theorem.4 states that in this case 4 g + 4 g is the best approximation of s as a linear combination of g and g, in the sense that it minimizes the -norm of the error. Example.8. In addition to the signals g n and g n defined in Example.7, define the signals g n and g 3 n as follows: jπn g n exp, n,, 3, 4; 4 jπ3n and g 3 n exp, n,, 3, 4. 4 In other words, we now have four signals, g k n, k,,, 3, defined for n,, 3, 4 by: jπkn g k n exp. 4 Similarly to Example.7, it is easy to check that these four signals are pairwise orthogonal. Since they are all nonzero, Theorem. implies that they form an orthogonal basis for C 4. Theorem.3 is therefore applicable to any signal in C 4, in particular, both to s and s defined in Example.7. a Using Theorem.3, find coefficients a,a,a,a 3 in the following Fourier series expansion: sn a g n+a g n+a g n+a 3 g 3 n, n,, 3, 4, for signal sn defined in Example.7.

20 Sec..3. Frequency Analysis 5 b Using Theorem.3, find coefficients a,a,a,a 3 in the following Fourier series expansion: s n a g n+a g n+a g n+a 3g 3 n, n,, 3, 4, for signal s n defined in Example.7. Solution. a We already know from Example.7 that s g +g i.e., the additional basis signals g and g 3 are not needed to represent s. The answer is a a 3 and a a. If we did not have the results of Example.7 available to us, we would proceed similarly to Example.7. First, we write all signals as vectors: g, g j, g, g 3 j, s +j. j j j Then we calculate the inner products used in Eq.., and compute the coefficients. These calculations were done in Example.7 for g and g. The calculations for g and g 3 are similar: g, g g 3, g 3 4, and s, g + +j + + j, a s, g g, g 4, s, g 3 + +j j + + j j + +jj + j j j + j + j + j, a 3 s, g 3 g 3, g 3 4. b We can use g k, g k 4, computed in Example.7 and Part a above. Recall that s. This makes its inner products with the basis vectors very simple: a k s, g k g k, g k g k + g k + g k 3 + g k 4 4 g k3 { 4 /4, k,, /4, k, 3.

21 5 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS Therefore, s n 4 g n 4 g n+ 4 g n 4 g 3n. This is consistent with what we saw in Example.7: s cannot be represented as a linear combination of only g and g. Note, however, that if in the expansion s 4 g 4 g + 4 g 4 g 3, we drop the terms which do not contain g and g, we will get the following vector: 4 g + 4 g, which is the answer we obtained in Example.7. This is the closest approximation of s by a linear combination of g and g. Now let us generalize Examples.7 and.8 from four dimensions to N. Example.9 Discrete Fourier Transform. Consider the following DT complex exponential functions: g k n jπkn N exp, n,...,n ; k,...,n..4 N In other words, there are N functions, g n,g n,...,g N n, and each of them is defined for n,,...,n. a Prove that these N signals are pairwise orthogonal, and find their energies. b Find a formula for the Fourier series coefficients X,X,...,XN of an N-point complex-valued signal xn, xn N N k N Xkg k n, n,...,n k jπkn Xk exp N, n,...,n..5 Solution. a To show orthogonality and compute the energies, we need to calculate all inner products g k, g i, for all k,...,n and i,...,n. If we can show that these inner products for k i are zero, we will show that the signals are pairwise orthogonal. Moreover, the inner products for k i will give us the energies. g k, g i N n g k ng i n

22 Sec..3. Frequency Analysis 53 N n N exp N N N n N n exp [ exp jπkn N N exp jπin N jπk in N ] jπk i n. N When k i, each term of the summation is equal to, and therefore the sum is N. The energy of each g k is therefore N/N /N. When k i, the sum is zero why?. b Since g,...,g N are nonzero and pairwise orthogonal, we can apply Theorem. to infer that {g,...,g N } is an orthogonal basis for C N. Any signal s C N can therefore be uniquely represented as their linear combination, according to Theorem.3. The coefficients in the representation are given by Eq... The denominator of that formula is the energy of g k, which we found in Part a to be /N. Therefore, Xk x, g k g k, g k x, g k /N N N N n N n N n xng k n xn N exp jπkn N xn exp jπkn, k,...,n..6 N The representation.5 of an N-point complex-valued signal x as a linear combination of complex exponentials.4 of frequencies, π/n,...,πn /N is called the DT Fourier series for the signal x: xn N N k jπkn Xk exp N, n,...,n..7 The signal X comprised of the N Fourier series coefficients is called the discrete Fourier transform DFT of the signal x. The DFT Xk is obtained from xn as follows: Xk N n xn exp jπkn, k,...,n..8 N

23 54 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS Since Eq..7 is the recipe for obtaining signal samples xn from the DFT, it is sometimes called the inverse DFT formula. Eqs..7 and.8 are particular cases of Eqs.. and., respectively, and were easily obtained in Example.9 by applying Theorem.3 to a complex exponential basis also called Fourier basis for C N. Eq..7 tells us how to represent any signal in C N as the linear combination of Fourier basis functions. The k-th term in the representation is the orthogonal projection of the signal onto the k-th basis signal. The projection coefficients are calculated using Eq..8. Note that Example.8 is a special case of Example.9: by setting N 4in Eqs..7 and.8 and appropriately normalizing the basis functions, we can get the answers to Example Complex Exponentials as Eigenfunctions of LTI Systems; Discrete- Time Fourier Transform DTFT One of the major reasons for the importance of Fourier series representations and also Fourier transforms is that the Fourier basis functions are eigenfunctions of LTI systems. In other words, the response of an LTI system to any one of the Fourier basis functions of Eq..4 g k is that same function times a complex number, α k g k, where the complex number α k depends on the system and on the frequency of the Fourier basis function. To see this, let us consider an LTI system with the impulse response h, and use the DT convolution formula to calculate its response to the following signal: xn e jωn for all integer n, where the frequency ω is an arbitrary fixed real number. We have that the response y is: yn hmxn m m m e jωn hme jωn m m hme jωm e jωn He jω, where He jω hme jωm. m The function H, viewed as a function of ω, is called the frequency response of the system, and is the discrete-time Fourier transform DTFT, not to be confused with DFT of the impulse response h. We will have much more to say about DTFTs below. For now, it is important to note that we obtained the following property.

24 Sec..3. Frequency Analysis 55 Theorem.5. Consider an LTI system whose impulse response is h. If the complex exponential signal e jωn, defined for all integer n, is its input, then the output is He jω e jωn, for all integer n, where He jω is the frequency response of the system evaluated at the frequency ω of the input signal. Example.. a Find a difference equation that describes the system in Fig..7, i.e., relates the output of the overall system to its input. Here, A, B, and C are fixed constants. xn Delay by Delay by B A C yn + Figure.7. The system diagram for Example.. Solution. From the system diagram of Fig..7, we have: yn Axn+Bxn + Cxn. b Find the frequency response of this system by calculating the response to a complex exponential. Solution. To find He jω, we calculate the response yn of the system to the input signal e jωn, xn e jωn yn Ae jωn + Be jωn + Ce jωn A + Be jω + Ce jω e jωn. }{{} He jω c Suppose that A B C and xn 5, for <n<. Calculate yn using the frequency response. Solution. xn 5e jn yn He j 5e jn , for all integer n.

25 56 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS Example.. Consider a DT LTI system with a known frequency response He jω. The response of such a system to an everlasting complex exponential input signal gn e jωn, <n<, is: yn gnhe jω, for <n<..9 In other words, e jωn is an eigenfunction of the system with an eigenvalue He jω. a Does Eq..9 hold for sinusoidal inputs? In other words, suppose that the input to a DT LTI system is cos ωn + ϕ, for <n<. Is it always the case that the output is cos ωn + ϕhe jω, for <n<? Does this hold for DT LTI systems with even frequency responses, i.e., for systems with He jω He jω? Solution. We decompose cosωn + ϕ into complex exponentials, cosωn + ϕ {e jωn+ϕ + e jωn+ϕ} ejϕ e jωn + e jϕ e jωn yn ejϕ He jω e jωn + e jϕ He jω e jωn. Unless He jω is even, i.e., He jω He jω, or else yn is not equal to cosωn + ϕhe jω. Note: in general, cosωn + ϕ and e jωn un, are not eigenfunctions of LTI systems. b Use the above property.9 to derive a simple expression for the response of the system to the following input signal: cos 8n + π 4. Solution. Using the previous part, we get: { yn ej π 4 He j8 e j8n + } e j π 4 He j8 e j8n He j8 e j8n+ π 4 + He j8 e j8n+ π Further Remarks on the Importance of DFT Suppose that S is an LTI system with a known impulse response h of duration N: hn for n,...,n hn otherwise. Suppose further that x is an input signal which is periodic with period N. How many arithmetic operations will it take to calculate the response y using the convolution formula? First, notice that the response is also periodic with period N: yn+n m hmxn+n m m hmxn m yn, for any integer n.

26 Sec..3. Frequency Analysis 57 We therefore only need to calculate N samples of yn, for example, for n,...,n. Since h has duration N and x has infinite duration, the convolution sum will always have N terms: yn m hmxn m N m hmxn m hxn+hxn hn xn N +, for n,...,n. Therefore, the computation of one particular sample yn requires N multiplications and N additions. 5 There are altogether N samples of y to be computed, and therefore the overall number of multiplications is N and the overall number of additions is NN N N. The overall order of the number of operations required is N. In other words, the process of calculating the response has computational complexity ON. This is a very high computational cost for something as basic and as frequently needed as calculating a discrete-time convolution. Specifically, suppose that on our computer, this operation takes second for N N. Then for N N it will be approximately times slower, i.e., it will take approximately.8 hours. We need an algorithm for calculating convolutions more quickly. Let us use the fact that, since x is periodic with period N, it can be represented as the following linear combination: x N k Xkg k, where g k are the Fourier basis functions given by Eq..4. Using the linearity of our system, we obtain the following representation for the output: y S[x] N k XkS[g k ]. But since g k is a complex exponential of frequency πk/n, we can use Theorem.5 to write the response to g k as S[g k ]He jπk/n g k. Substituting this into the formula for y, we get: y S[x] N k XkHe jπk/n g k. But this is a representation of the signal y as a linear combination of Fourier basis signals g k, with coefficients XkHe jπk/n. In other words, the Fourier series coefficients of y are: Y k XkHe jπk/n, for k,...,n. Assuming that the values of He jπk/n for k,...,n are known, we have discovered the following procedure for calculating y: 5 These are complex multiplications and complex additions since we assume in general that our signals are complex-valued.

27 58 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS Step. Calculate the N DFT coefficients Xk ofx using the DFT formula. Step. Calculate the N DFT coefficients Y k XkHe jπk/n ofy. Step 3. Calculate y from its DFT coefficients using the inverse DFT formula. It turns out that Steps and 3, the DFT and the inverse DFT, can be implemented using a fast Fourier transform FFT 6 algorithms whose computational complexity is ON log N. Moreover, if the N values He jπk/n of the frequency response were not known but had to be calculated from the impulse response h, that, too, could be done using FFT with computational complexity ON log N. Step consists of N multiplications. The overall computational complexity is therefore only ON log N, a marked improvement over ON, especially for large N. To summarize, we have just seen two aspects of why it is important to study representations of signals as weighted sums of complex exponentials in the context of our study of LTI systems. Conceptual importance: LTI systems process each frequency component separately, and in a very simple way i.e. by multiplying it by a frequency-dependent complex number. Computational importance: To obtain the response of an LTI system with an N-point impulse response to an N-periodic signal, we need ON computational effort if we use the convolution formula directly. The computational complexity is, however, reduced to ON log N if we use the frequency-domain representations instead..3.6 Continuous-Time Fourier Series As indicated above, the notions of orthogonal bases and projections can be extended to spaces of CT signals. Determining whether a series representation converges and if so, what it converges to is much more complicated than for finite-duration DT signals. We therefore will only consider one very important example CT Fourier series. Consider the set of signals L defined as follows: it is the set of all periodic complex-valued CT signals st with period for which τ+t τ st dt <, where τ is an arbitrary real number i.e., the integral is taken over any period. It turns out that L is a vector space each vector in this case being a continuous-time signal. If we define the inner product of two functions as follows: s, g τ+t τ stgt dt, 6 We will discuss the mechanics of FFT below. What is important for now is that it is a fast algorithm for calculating the DFT of a signal.

28 Sec..3. Frequency Analysis 59 then the infinite collection of -periodic complex exponentials jπkt g k t exp, k, ±, ±,... forms an orthogonal basis for L. We can represent any -periodic CT signal s as a linear combination of these complex exponentials: st k a k g k t k jπkt a k exp.. The first sign in Eq.. needs careful interpretation: unlike the finite-duration DT case, the equality here is not pointwise. Instead, the equality is understood in the following sense: M s a k g k as N and M. kn Nevertheless, formula. is still valid: The inner product of g k and g i is easily computed: g k,g i τ+t τ τ+t τ a k s, g k g k,g k.. jπkt exp exp jπit jπk it exp dt { T if k i if k i dt The fact that g k,g i shows that our vectors are indeed pairwise orthogonal. 7 Substituting g k,g k back into Eq.., we get: a k s, g k τ+t st exp jπkt dt.. τ 7 Note, however, that this is not enough to prove that they form an orthogonal basis for L since L is infinite-dimensional, and Theorem. no longer holds: it is not true that any infinite set of nonzero orthogonal vectors forms an orthogonal basis for L. Proving the fact that signals g k do form an orthogonal basis for L is beyond the scope of this course.

29 6 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS st A t t t Figure.8. Signal st of Example.. Example.. Let, t, and A be three positive real numbers such that >t >. Consider the following periodic signal: A if t t st { if t t, periodically extended with period, as shown in Fig..8. τ /, its Fourier series coefficients are: a t / Adt At, t / a k t / A exp jπkt dt t / jπkt A jπk exp A [ πk j jπkt exp A πk sin πkt. t / exp t / jπkt Note that this last formula is also valid for k if we define sin θ θ Using Eq.. with ]. θ Another common way of decomposing CT periodic signals as linear combinations of sinusoidal signals is by using sines and cosines as basis functions, instead of complex exponentials. The following infinite collection of functions is also an orthogonal basis for L, and is also called a Fourier basis: c t, πkt c k t cos, k,,... πkt s k t sin, k,,...

30 Sec..3. Frequency Analysis 6 As we did previously, let us first prove that these functions are pairwise orthogonal, and find their energies over one period. We need to consider all pairwise inner products which will now be integrals of products of trigonometric functions. We will therefore need the following formulas: sin α sin β cosα β cosα + β.3 sin α cos β sinα β + sinα + β.4 cos α cos β cosα β + cosα + β.5 Now we compute the inner products, keeping in mind that s k t is defined for k while c k t is defined for k : s k,s i s k,c i Eq..3 Eq..4 c k,c i Eq..5 τ+t τ τ+t sin πk t τ { T, k i, k i. τ+t τ τ+t τ τ+t τ τ+t τ sin πi t [ cos πk i t sin πk t cos πi t [ sin πk i t cos πk t cos + sin cos πi t [ cos πk i t, k i, k i, k i. dt dt πk + i t ] πk + i t ] + cos dt πk + i t ] dt dt. We are now ready to derive formulas for the coefficients a,a...and b,b,b...of the expansion of a CT -periodic signal st: b s, c c,c st b + a k s k t+ b k c k t..6 k k dt

31 6 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS xt t Figure.9. Signal xt of Example.3. τ+t st dt b k s, c k c k,c k τ+t a k s, s k s k,s k τ+t τ τ τ st cos πk t st sin πk t dt, k,,... dt, k,,... Example.3. Suppose that the period is, and let signal x be defined by: {, t< xt, t<, as illustrated in Fig..9 Let us compute the Fourier series coefficients with respect to the Fourier basis of sines and cosines. From the formulas above, with τ, b For k, b k dt cos πk t dt cosπkt dt πk sinπkt t t a k sin πk t dt sinπkt dt

32 Sec..3. Frequency Analysis 63 πk cosπkt t t { πk, if k is odd, if k is even. A different method of finding the coefficients is to notice that a Signal x is related to signal s of Example.. If we set t, A, and in Example.3 and shift s by t / to the left, we will obtain x: xt st + t. b Complex exponential basis signals are related to the sine and cosine basis signals. Using the Fourier series coefficients obtained in Example., we have the following representation of xt in the complex exponential Fourier basis: xt s t + t A πk sin πkt g k t + t k A πk sin πkt jπkt + t / exp k k k A πk sin πkt jπkt / jπkt exp exp A πk sin πkt jπkt exp g k t, which means that the coefficients of x in the complex exponential Fourier basis are: α k A πk sin πkt jπkt exp πk jπk πk sin exp..7 But notice that the complex exponential basis functions are related to the sine and cosine basis functions as follows: g t c t, πkt πkt For k, g k t cos + j sin c k t+js k t, g k t cos πkt + j sin πkt c k t js k t.

33 64 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS Therefore, xt k α g t+ α g t+ α c t+ α k g k t α k g k t+ k k α k g k t [α k g k t+α k g k t] k [α k c k t+js k t + α k c k t js k t] k α c t+ α k + α k c k t+ jα k α k s k t. k Matching these coefficients with the coefficients in Eq..6, we get the following relationship between the exponential Fourier series coefficients and the sine-cosine Fourier series coefficients of any -periodic signal: k b α, b k α k + α k for k,,..., a k jα k α k for k,,... Using expression.7 we found for the coefficients α k of the specific signal x we are considering in this example, we get: b α [ ] sinπk/ jπk πk/ exp k ; b k α k + α k πk jπk πk sin exp + π k sin πk exp jπk πk jπk πk sin exp + exp jπk πk πk πk sin cos Eq..4 sin + sin πk ; πk πk sin πk jπk a k jα k α k j exp j πk jπk πk sin exp exp jπk πk πk sin π k sin πk exp jπk

34 Sec..3. Frequency Analysis 65 { πk, if k is odd, if k is even. Notice that these are the same results we got before by evaluating the inner products with sines and cosines directly..3.7 Discrete-Time Fourier Transform DTFT, Revisited The definition of the discrete-time Fourier transform of a discrete-time signal xn is: Xe jω n xne jωn. Notice that xn are the continuous-time Fourier series coefficients of Xe jω. To see this better, we can relate DTFT to the CTFS formulas above by making the following identifications: π, ω πt/ t, n k, xn a k. The DTFT formula then becomes the inverse CTFS formula., and therefore the inverse DTFT formula is obtained by relabeling variables in the CTFS formula.: xn ω +π Xe jω e jωn dω. π ω Thus, DTFS/DFT, CTFS, and DTFT are all particular cases of our general framework of orthogonal representations. Properties of the DTFT.. Linearity: the DTFT of a x n+a x n is. Delay: the DTFT of x n n is n a X e jω + a X e jω. x n n e jωn n e jωn mn n n xme jωm } {{ } Xe jω X e jω e jωn. e jωn

35 66 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS 3. Convolution: Y e jω property yne jωn n [ ] hn kxk n k e jωn [ ] hn ke jωn xk k k n [ H e jω e jωk] xk H e jω e jωk xk k } {{ } Xe jω H e jω X e jω So, if yn h xn, then Y e jω H e jω X e jω. 4. Parseval s theorem: n n xn π X e jω dω π π xny n π π π X e jω Y e jω dω X e j n xn x π X e jω dω π π 7. Modulation: the DTFT of xne jωn is xne jω ωn X n e jω ω. Example.4. yn xn+xn Find the frequency response He jω. Is this system a low-pass, band-pass, or high-pass filter? Plot H e jω and H e jω.

36 Sec..3. Frequency Analysis 67 Solution. Method. Use the eigenfunction property: xn e jωn yn H e jω e jωn yn e jωn + e jωn Method. + e jω }{{} He jω H e jω e j ω e j ω cos ω e jωn e j ω + e j ω Y e jω { } DTFT xn+xn DTFT{xn} + DTFT{xn } X e jω + X e jω e jω X e jω +e jω H e jω Y e jω X e jω +e jω Method 3. Impulse response: hn δn+δn H e jω DTFT{hn} DTFT{δn} δne jωn n DTFT{δn } e jω e jω H e jω +e jω To plot the magnitude response and the phase response, we note: H e jω cos ω H e jω e j ω }{{} + cos ω, ω }{{}

37 68 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS because cos ω for π ω π, i.e., π ω π. Note: H jω is periodic with period π, since it is a DTFT of a DT signal. Since the values of H e jω at low frequencies near ω are close to one, and its values at high frequencies near ω ±π are close to zero, this a low-pass filter. H `ejω π π ω Figure.3. The plot of H `e jω, for Ex..4. H `ejω π π π π ω Figure.3. The plot of H `e jω, for Ex..4. Example.5. Repeat Ex..4 for the following system: Solution. yn x xn. hn δn δn H e jω e jω [ je jω e jω e jω] je jω sin ω j }{{} sin ω

38 Sec..3. Frequency Analysis 69 because and therefore, H e jω sin ω H e jω j + }{{}} e{{ jω } + sin ω π ω { π ω, ω π ω, π ω, π {, ω π sin ω, π ω, sin ω {, ω π π, π ω. This is a band-pass filter. Note: it is a convention to keep angles in the range [ π, π]. H `ejω π π π π ω Figure.3. The plot of H `ejω, for Ex..5. H `ejω π π π π π ω π Figure.33. The plot of H `e jω, for Ex Discrete Fourier Transform DFT, Revisited Our motivation for studying frequency analysis of DT signals is summarized in Fig..34.

39 7 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS N X xn Xkg k n k He jω N X yn XkH k e j πk N g k n Figure.34. A system diagram motivating the use of frequency analysis. In this figure, the input xn isn-periodic and the basis function g k n is given by g k n N ej πk N n for k,,n and for all n..8 Conceptual importance: LTI systems process each harmonic separately in a simple way i.e., multiply each harmonic by a frequency dependent complex number g k n. Computational importance: to obtain XkH e j πk N from Xk for k,,,n, we need only N operations. to obtain Xk from xn and yn from XkH e j πk N, we need only ON log N operations. This can be done through FFT. If we put xn and g k n in vector form, we get πk x N ej N x πk x., g k N ej N. xn N We can then write N x πk ej N N k for k,,,n..9 Xkg k..3 Since the g k s are pairwise orthogonal, we can use the projection formula to calculate the coefficients and obtain the inversion formula.8. Remark. DFT DTFT although related. DFT: discrete frequency k. DTFT: continuous frequency ω.

40 Sec..3. Frequency Analysis 7 Remark. e j πk N n is periodic as a function of n, with period N. Therefore, IDFT defines a periodic signal for <n<. We will often think of xn asn-periodic. If we let then we can rewrite Eq..3 as x N k X Xkg k X. XN Xg + Xg + + XN g N X g g g N. XN g g g N X BX, where B is an N N matrix whose columns are g k s and whose entry B nk in the n-th row and k-th column is given by B nk N πk n ej N for n,,,n; k,,,n. To get the formula for DFT, we premultiply x BX by the matrix A NB H, where y H y T means the conjugate transpose of y. Since the rows of B H are conjugate transposes of g k s, we have g H g H A N., g H N where gk H πk N e j N πk N e j N πk N e j N N. The entry in the k-th row and n-th column of A is: A kn NB nk πk n e j N for k,,,n; n,,,n. Premultiplying x BX by A, we get Ax ABX.

41 7 CHAPTER. ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT SYSTEMS X A x N N N N Figure.35. An illustration of the DFT as matrix multiplication. We now calculate AB. AB N g H g H. g g g N N g H N g Hg g Hg g Hg N g Hg g Hg g Hg N..... gn H g gn H g gn H g N Note that since AB simplifies to AB N g H k g p g p, g k N N n N g p ng k n,... N... I, where I is the identity matrix.

42 Sec..3. Frequency Analysis 73 Hence, A and B are inverses of each other. The DFT of x can then be calculated by X Ax. From Fig..35, we see that we need N complex multiplications for a brute force implementation of DFT. However, the fact that the matrix A is highly structured can be exploited to produce a much faster algorithm for multiplying a vector by this matrix A.