1 Signals and systems

Transcription

1 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis Signals and systems In the first two chapters we will consider some basic concepts and ideas as the mathematical background for the specific discussions of the various orthogonal transform methods in the subsequent chapters. Here, we will set up a framework common to all such methods, so that they can be studied from a unified point of view. While some discussions here may seem mathematical, the emphasis is on the intuitive understanding, rather than the theoretical rigor.. Continuous and discrete signals A physical signal can always be represented as a real- or complex-valued continuous function of time x(t) (unless specified otherwise, such as a function of space). The continuous signal can be sampled to become a discrete signal x[n]. If the time interval between two consecutive samples is assumed to be, then the nth sample is x[n] =x(t) t=n = x(n ). (.) In either the continuous or discrete case, a signal can be assumed in theory to have infinite duration; i.e., <t< for x(t) and <n< for x[n]. However, any signal in practice is finite and can be considered as the truncated version of a signal of infinite duration. We typically assume t T for a finite continuous signal x(t), and n N (or sometimes n N for certain convenience) for a discrete signal x[n]. The value of such a finite signal x(t) isnot defined if t<ort>t; similarly, x[n] is not defined if n<orn>n. However, for mathematical convenience we could sometimes assume a finite signal to be periodic; i.e., x(t + T )=x(t) andx[n + N] =x[n]. A discrete signal can also be represented as a vector x =[...,x[n ],x[n],x[n +],...] T of finite or infinite dimensions composed of all of its samples or components as the vector elements. We will always represent a discrete signal as a column vector (transpose of a row vector).

2 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis 2 Signals and systems Definition.. The discrete unit impulse or Kronecker delta function is defined as { n = δ[n] = n. (.2) Based on this definition, a discrete signal can be represented as x[n] = m = x[m]δ[n m], (n =, ±, ±2,...). (.3) This equation can be interpreted in two conceptually different ways. First, a discrete signal x[n] can be decomposed into a set of unit impulses each at a different moment n = m and weighted by the signal amplitude x[m] at the moment, as shown in Fig..(a). Second, the Kronecker delta δ[n m] acts as a filter that sifts out a particular value of the signal x[n] at the moment m = n from a sequence of signal samples x[m] for all m. This is the sifting property of the Kronecker delta. Figure. Sampling and reconstruction of a continuous signal. In a similar manner, a continuous signal x(t) can also be represented by its samples. We first define a unit square impulse function as { / t< δ (t) =. (.4) else

3 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis Signals and systems 3 Note that the width and height of this square impulse are respectively and / ; i.e,itcoversaunitarea / =, independent of the value of : δ (t) dt = / =. (.5) Now a continuous signal x(t) can be approximated as a sequence of square impulses δ (t n ) weighted by the sample value x[n] for the amplitude of the signal at the moment t = n : x(t) ˆx(t) = x [n] δ (t n ). (.6) n= This is shown in Fig..(b). The approximation ˆx(t) above will become a perfect reconstruction of the signal if we take the limit, so that the square impulse becomes a continuous unit impulse or Dirac delta: which is formally defined as lim δ (t) =δ(t). (.7) Definition.2. The continuous unit impulse or Dirac delta function δ(t) is a function that has an infinite height but zero width at t =, and it covers a unit area; i.e., it satisfies the following two conditions: δ(t) = { t = t and δ(t) dt = + δ(t) dt =. (.8) Now at the limit, the summation in the approximation of Eq. (.6) above becomes an integral, the square impulse becomes a Dirac delta, and the approximation becomes a perfect reconstruction of the continuous signal: x(t) = lim x[n]δ (t n ) = x(τ)δ(t τ) dτ. (.9) n= In particular, when t =, Eq. (.9) becomes x() = x(τ)δ(τ) dτ. (.) Equation (.9) can be interpreted in two conceptually different ways. First, a continuous signal x(t) can be decomposed into an uncountably infinite set of unit impulses each at a different moment t = τ, weighted by the signal intensity x(τ) at the moment t = τ. Second, the Dirac delta δ(τ t) acts as a filter that sifts out the value of x(t) at the moment τ = t from a sequence of uncountably infinite signal samples. This is the sifting property of the Dirac delta.

4 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis 4 Signals and systems Note that the discrete impulse function δ[n] has a unit height, while the continuous impulse function δ(t) has a unit area (product of height and width for time); i.e., the two types of impulses have different dimensions. The dimension of the discrete impulse is the same as that of the signal (e.g., voltage), while the dimension of the continuous impulse is the signal s dimension divided by time (e.g., voltage/time). In other words, x(τ)δ(t τ) represents the density of the signal at t = τ, only when integrated over time will the continuous impulse functions have the same dimension as the signal x(t). The results above indicate that a time signal, either discrete or continuous, can be decomposed in the time domain to become a linear combination, either a summation or an integral, of a set of time impulses (components), either countable or uncountable. However, as we will see in future chapters, the decomposition of the time signal is not unique. The signal can also be decomposed in different domains other than time, such as frequency, and the representations of the signal in different domains are related by certain orthogonal transformations..2 Unit step and nascent delta functions Here we define some important functions to be used frequently in the future. The discrete unit step function is defined as Definition.3. u[n] = { n n<. (.) The Kronecker delta can be obtained as the first-order difference of the unit step function: { n = δ[n] =u[n] u[n ] = n. (.2) Similarly, in continuous case, the impulse function δ(t) is also closely related to the continuous unit step function (also called Heaviside step function) u(t). To see this, we first consider a piece-wise linear function defined as t< u (t) = t/ t<. (.3) t Taking the time derivative of this function, we get the square impulse considered before in Eq. (.4): δ (t) = d t< dt u (t) = / t<. (.4) t

5 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis Signals and systems 5 If we let, then u (t) becomes the unit step function u(t) at the limit: Definition.4. t< u(t) = lim u (t) = /2 t =. (.5) t> Here, we have defined u() = /2 att = for reasons to be discussed in the future. Also, at the limit, δ (t) becomes Dirac delta discussed above: { t = δ(t) = lim δ (t) = t. (.6) Therefore, by taking the limit on both sides of Eq. (.4) we obtain a useful relationship between u(t) andδ(t): d u(t) =δ(t), u(t) = dt t δ(τ) dτ. (.7) This process is shown in the three cases for different values of in Fig..2. Figure.2 Generation of unit step and unit impulse. Three functions u (t) with different values of together with their derivatives δ (t) areshown.in particular, when δ, these functions become u(t) andδ(t), as shown on the right. In addition to the square impulse δ (t), the Dirac delta δ(t) can also be generated from a variety of different nascent delta functions at the limit when a certain parameter of the function approaches the limit of either zero or infinity. Consider, for example, the Gaussian function: g(t) = 2πσ 2 e t2 /2σ 2, (.8) which is the probability density function of a normally distributed random variable t with zero mean and variance σ 2. Obviously the area underneath this Although in some of the literature it could be alternatively defined as either u() = or u() =.

6 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis 6 Signals and systems density function is always one, independent of σ: g(t) dt = 2πσ 2 e t2 /2σ 2 dt =. (.9) At the limit σ, this Gaussian function g(t) becomes infinity at t = but it is zero for all t ; i.e., it becomes the unit impulse function: lim σ 2πσ 2 e t2 /2σ 2 = δ(t). (.2) The Gaussian functions with three different σ values are shown in Fig..3. Figure.3 Gaussian functions with different σ values (.5,, 2). The argument t of a Dirac delta δ(t) may be scaled so that it becomes δ(at). In this case Eq. (.) becomes x(τ)δ(aτ) dτ = ( u ) x δ(u) a a du = x(), (.2) a where we have defined u = aτ. Comparing this result with Eq. (.), we see that δ(at) = δ(t); i.e. a δ(at) =δ(t). (.22) a For example, a delta function δ(f) of frequency f can also be expressed as a function of angular frequency ω = 2πf as δ(f) =2πδ(ω). (.23) More generally, the Dirac delta can also be defined over a function f(t) ofa variable, instead of a variable t. Now the Dirac delta becomes δ(f(t)), which is zero except when f(t k ) =, where t = t k is one of the roots of f(t). To see how such an impulse is scaled, consider the following integral: x(τ)δ(f(τ)) dτ = x(τ)δ(u) f du, (.24) (τ)

7 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis Signals and systems 7 where we have changed the integral variable from τ to u = f(τ). If τ = τ is the only root of f(τ); i.e., u = f(τ ) =, then the integral above becomes If f(τ) has multiple roots τ k, then we have x(τ)δ(f(τ)) dτ = x(τ ) f (τ ). (.25) x(τ)δ(f(τ)) dτ = k x(τ k ) f (τ k ). (.26) This is the generalized sifting property of the impulse function. We can now express the delta function as δ(f(t)) = k δ(t t k ) f (τ k ), (.27) which is composed of a set of impulses each corresponding to one of the roots of f(t), weighted by the reciprocal of the absolute value of the derivative of the function evaluated at the root..3 Relationship between complex exponentials and delta functions Here we list a set of important formulas that will be used in the discussions of various forms of the Fourier transform in Chapters 3 and 4. These formulas show that the Kronecker and Dirac delta functions can be generated as the sum or integral of some forms of the general complex exponential function e j2πft = e jωt. The proofs of these formulas are left as homework problems. I. Dirac delta as an integral of a complex exponential: e ±j2πft dt = cos(2πft) dt ± j sin(2πft) dt =2 cos(2πft) dt = δ(f) =2πδ(ω). (.28) Note that the integral of the odd function sin(2πft) over all time <t< is zero, while the integral of the even function cos(2πft) over all time is twice the integral over <t<. Equation (.28) can also be interpreted intuitively. The integral of any sinusoid over all time is always zero, except if f =ande ±j2πft =, then the integral becomes infinity. Alternatively, if we integrate the complex exponential with respect to frequency f, weget e ±j2πft df =2 cos(2πft) df = δ(t), (.29) which can also be interpreted intuitively as a superposition of uncountably infinite sinusoids with progressively higher frequency f. These sinusoids cancel

8 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis 8 Signals and systems each other at any time t except if t = and cos(2πft) = for all f, then their superposition becomes infinity. Ia. This formula is a variation of Eq. (.28): e ±j2πft dt = Given the above, we can also get: e ±jωt dt = e ±jωt dt = 2 δ(f) j2πf = πδ(ω) jω. (.3) e ±jωt d( t) = e jωt dt = 2 δ(f) ± j2πf = πδ(ω) ± jω. (.3) Adding the two equations above we get the same result as given in Eq. (.28): e ±jωt dt = e ±jωt dt + e ±jωt dt = δ(f) =2πδ(ω). (.32) II. Kronecker delta as an integral of a complex exponential: e ±j2πkt/t dt = cos(2πkt/t) dt ± j sin(2πkt/t) dt T T T T T T = cos(2πkt/t) dt = δ[k]. (.33) T In particular, if T =wehave T e ±j2πkt dt = δ[k]. (.34) III. A train of Dirac deltas with period F as a summation of a complex exponential: F = F n= n= e ±j2πfn/f = F cos(2πfn/f )= In particular, if F =wehave n= e ±j2πfn = n= k= k= cos(2πfn/f ) ± j F δ(f kf) = δ(f k) = IIIa. This formula is a variation of Eq. (.36): e ±j2πfn = 2 n= k= δ(f k)+ k= e ±j2πf = k= k= n= sin(2πf n/f ) 2πδ(ω 2πkF).(.35) 2πδ(ω 2πk). (.36) πδ(ω 2πk)+ e ±jω. (.37)

9 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis Signals and systems 9 Given the above, we can also get n= e ±j2πfn = = 2 e j2πfn = 2 n= k= δ(f k) k= δ(f k)+ e j2πf. (.38) e ±j2πf Adding the two equations above we get the same result as given in Eq. (.36): n= e ±j2πfn = = n= k= e ±j2πfn + n= δ(f k) =2π e ±j2πfn k= δ(ω 2πk). (.39) IV. A train of Kronecker deltas with period N as a summation of complex exponential: N e ±j2πnm/n = N N n= = N N n= N n= cos(2πnm/n) ± j N cos(2πnm/n) = N n= k= sin(2πnm/n) δ[m kn]. (.4).4 Attributes of signals A time signal can be characterized by the following parameters. The energy contained in a continuous signal x(t) is E = or in a discrete signal x[n], it is E = n= x(t) 2 dt, (.4) x[n] 2. (.42) Note that x(t) 2 and x[n] 2 have different dimensions and they represent respectively the power and energy of the signal at the corresponding moment. If the energy contained in a signal is finite E <, then the signal is called an energy signal. A continuous energy signal is said to be square-integrable, and a discrete energy signal is said to be square-summable. All signals to be considered in the future, either continuous or discrete, will be assumed to be energy signals.

10 Introduction to Orthogonal Transforms: With Applications in Data Processing and Analysis Signals and systems The average power of the signal is T P = lim x(t) 2 dt, (.43) T T or for a discrete signal, it is P = lim N N N x[n] 2. (.44) If E of x(t) is not finite but P is, then x(t) isapower signal. Obviously, the average power of an energy signal is zero. The cross-correlation defined below measures the similarity between two signals as a function of the relative time shift: r xy (τ) =x(t) y(t) = n= x(t) y(t τ) dt = x(t + τ) y(t) dt x(t τ) y(t) dt = y(t) x(t) =r yx (τ). (.45) Note that the cross-correlation is not commutative. For a discrete signal, we have r xy [m] =x[n] y[n] = x[n] y[n m] = x[n + m] y[n]. (.46) n= n= In particular, when x(t) = y(t) andx[n] = y[n], the cross-correlation becomes the autocorrelation, which measures the self-similarity of the signal: and r x (τ) = r x [m] = n= x(t)x(t τ) dt = x[n] x[n m] = n= x(t + τ)x(t) dt, (.47) x[n + m] x[n]. (.48) More particularly when τ =andm =wehave r x () = x(t) 2 dt r x [] = x[n] 2 dt, (.49) n= which represent the total energy contained in the signal. A random time signal x(t) is also called a stochastic process. Its mean or expectation is (Appendix B): µ x (t) =E[x(t)]. (.5) The cross-covariance of two stochastic processes x(t) and y(t) is Cov xy (t, τ) =σ 2 xy(t, τ) =E[(x(t) µ x (t)) (y(τ) µ y (τ))] = E[x(t)y(τ)] µ x (t)µ y (τ). (.5)