LING 439/539 notes: Digital signal processing

Transcription

1 LING 439/539 otes: Digital sigal processig These otes cover more details of DSP that are relevat to speech processig. I will cotiue to develop these otes, ad try to cover part of the materials i class. But you will ot be tested o all the details. However, if you do read these otes, ay commets are welcome! These otes are based o stadard sigal processig texts, such as: Oppeheim, A. V. ad R. W. Schafer Discrete-time sigal processig. Pretice-Hall, Upper Saddle River, NJ, 2d editio. Oppeheim, A. V. ad A. S. Willsky Sigals ad systems. Pretice Hall, Upper Saddle River, NJ, 2d editio. You ca also cosult these popular texts, which cotai similar materials: Lyos, R. G Uderstadig Digital Sigal Processig. Pearso Educatio. Proakis, J. G. ad D. K. Maolakis Digital Sigal Processig: Priciples, Algorithms ad Applicatios. Pretice-Hall, 3rd editio. 1 Sigals 1.1 Basic otatio A variety of thigs that chage over time ca be thought of as sigals (temperature, stock price, laguage, etc.). I mathematical terms, sigals are characterized as a fuctio that maps time to amplitude. Both time ad amplitude ca be discrete or cotiuous. As a example, speech sigals are obtaied by samplig a cotiuous sigal (air pressure variatio picked up by microphoe/ear). We will be dealig with 1-dimesioal sigals with either discrete or cotiuous time, ad cotiuous amplitude. The more commo otatio is to write x(), < < for discrete time (a two-sided sequece) ad x(t) for cotiuous time (a cotiuous fuctio). Probably the most importat sigal is the impulse: { 1, = 0 (1) δ() = 0, 0 1

2 The impulse i the cotiuous time is a little more difficult to defie. commo approach is to start with somethig like: { 1 (2) δ (t) =, 0 < t 0 otherwise. Takig the limit 0, we get the cotiuous impulse sigal δ(t). The uit step sigal is the same for discrete ad cotiuous sigals: { 1, 0 (3) u() = 0, < 0 Time shift: x( k) is the sigal x() delayed by k. How to represet u() i terms of δ()? How to represet δ() i terms of u()? I fact, a arbitrary sigal x() ca also be represeted i terms of δ(): The x() = x(k)δ( k) We will ofte use this otatio of summads. Here x() shows a while sequece, while x(k) shows specific istaces of x() at time k. The sum is take over all values of k, ad x() does ot deped o k. Sice all sequeces ca be represeted as liear combiatios of the impulse, we call δ() is a basis. The decompositio of a cotiuous sigal ito impulses is similar, by usig the followig approximatio of x(t): x(k )δ ( k ) Agai, by takig the limit 0, this leads to: x(t) = More examples of discrete sigals: x(s)δ(t s) ds (4) Real expoetial sigals: x() = a, 0. Defie eergy of a sigal: E(x()) = + = x() 2. Q: : Whe does a real expoetial sigal have fiite eergy? (5) Siusoidals: x() = A cos(2πω+φ), where A : amplitude, ω: frequecy, φ: phase. Siusoidals are examples of periodic sigals, sice for T = 1/ω, x() = x( + kt ), k = 1, 2,. Periodic sigals have very differet auditory percepts from aperiodic oes. For arbitrary periodic sigals, the smallest m such that x() = x( + m) is called a fudametal period. The fudametal period determies the perceptio of pitch. 2

3 1.2 Represetig sigals with complex umbers (6) Complex expoetial sigals: x() = e (r+jω) Complex umbers exted the real lie to the xy-plae. It is a coveiet aalytical tool for dealig with periodic quatities, such as siusoidal. While Math books ofte use i, egieers like to use j for the imagiary uit = 1. (7) Cartesia form: z = a + bj, where a, b R is also writte as a = Re(z), b = Im(z). Additio ad multiplicatio: (a + bj) + (c + dj) = (a + c) + (b + d)j; (a + bj) (c + dj) = (ac bd) + (bc + ad)j Complex cojugate: if z = a + bj, the z = a bj (sometimes z also used). Magitude: z 2 = z z. If z = a + bj, the z = a 2 + b 2. Q: : How to divide two complex umbers? (8) Euler s formula: e jx = cos x + j si x Polar form: z = z e j z. I the cartesia presetatio, the real umbers a ad b are called real ad imagiary part, respectively. I the polar represetatio, z is called amplitude, ad z is called phase. Q: : How to covert the cartesia form from/to polar form? Q: : How to represet a real siusoidal i terms of the complex umbers? The followig result has importat cosequeces i the aalysis of liear systems: (9) Fudametal theorem of algebra: ay polyomial with real coefficiets of degree has complex roots (ot ecessarily distict). Q: : What are the complex roots of z = 1? Q: : What does a complex expoetial sigal x() = e (r+jω) look like? 2 Liear Time-Ivariat (LTI) systems 2.1 Examples of systems Systems take iput sequeces to output sequeces. Usig T [.] to represet systems, a LTI system satisfies the followig properties: Causal systems : the system s output at = 0 oly depeds o the iput sequece values for 0. Stable systems : if the iput is bouded, the the output is also bouded. Oly stable ad causal systems ca be realized. Speech also falls i this category. 3

4 Liear systems : the output of ay liear combiatio of sequeces is the liear combiatio of their idividual outputs. For ay a, b R: T [ax 1 () + bx 2 ()] = at [x 1 ()] + bt [x 2 ()] Time-ivariat systems : if y() = T [x()], the y( k) = T [x( k)]. LTI system satisfies both liearity ad time-ivariace, ad is the most importat model for speech as well as the basis of almost all the sigal processig techiques that we will discuss. 2.2 Impulse respose The respose of the system to the impulse sigal is of particular importace i the aalysis of LTI systems. It is desigated a special symbol (for a fixed system T ): h() = def T [δ()] By combiig the impulse represetatio of a iput sigal x(), liearity, ad time-ivariace, we ca see why h() is a useful otio: y() = T [x()] = T [ = = x(k)δ( k)] x(k)t [δ( k)] x(k)h( k) I.e. the output sigal oly depeds o the iput ad h(). Ofte we will just use h() to represet the system, ad the above relatioship is called a covolutio: y() = x() h() Q: is covolutio commutative? i.e. is x() h() = h() x()? Q: what coditios should h() satisfy if the system is stable? Q: what coditios should h() satisfy if the system is causal? Q: what is the impulse respose of a cascade of two systems, h 1 () ad h 2 ()? 2.3 Systems described by liear differece equatios A special case of LTI systems is described by liear differece equatios: N M a k y( k) = b l x( l) k=0 4 l=0

5 Like the cotiuous differetial equatios, such equatios do ot uiquely determie a system, uless some other coditios are kow (e.g. iitial coditios). We do ot discuss how to solve these differece equatios, but ote the recursive formula that ca be used to derive y() oce some iitial values are kow: y() = N k=0 a k b 0 y( k) + M l=1 b l b 0 x( l) If a k = 0, k = 0,, N, the the system is called a fiite impulse respose (FIR) system. Otherwise, it s called a ifiite impulse respose (IIR) system. 3 Sigals ad systems i the Frequecy domai 3.1 Frequecy respose of a LTI system Give a system T, if a sequece x() satisfies T [x()] = λx(), i.e. we get essetially the same sigal back from T, the x() is called a eige sigal of T [.]. The importace of Fourier aalysis, is that it allows sequeces to be decomposed ito liear combiatio of siusoidals, which are eigefuctios of LTI systems. Let x() = e jw, < < (so we are cosiderig a steady-state respose). The: Writig + y() = = = h(k)x( k) h(k)e jw( k) = e jw + h(k)e jw e jwk h(k)e jwk h(k)e jwk as H(e jw ) = H(e jw ) e j H(ejw), we have y() = H(e jw )e jw. This shows that we get a sigal of the same frequecy i the output, ad the system oly modifies the amplitude ad phase of the sigal. Hece H(e jw ) is called a complex frequecy respose of the system. (10) Impulse respose: h() = a u(), a < 1. H(e jw ) ca be calculated directly. Some properties of H(e jw ): 5

6 H(e jw ) is a cotiuous fuctio of w. H(e jw ) is periodic with 2π as period. Therefore, the value of H(e jw ) is completely determied over ay iterval of legth 2π. Covetioally, this iterval is set to [ π, π]. If h() is real valued, the H(e jw ) is cojugate symmetric, i.e. H(e jw ) = H (e jw ). This poit shows that for real world systems (with real impulse respose), the real part of frequecy respose is eve: Re(H(e jw )) = Re(H(e jw )); while the imagiary part is odd: Im(H(e jw )) = Im(H(e jw )). The amplitude of frequecy respose is eve: H(e jw ) = H(e jw ) ; the phase of the frequecy respose is odd: H(e jw ) = H(e jw ). Q: how does a LTI system respod to a siusoidal sigal x() = A cos(ω+ φ)? Discrete time systems are also called digital filters. A example of a commoly used filter: (11) A ideal low pass filter is defied by its frequecy respose H(e jw ) = { 1, ω0 w ω 0 0, ω 0 < w < π 3.2 Fourier trasform pair The above defiitio of H(e jw ) illustrates a example of a Fourier trasform, i particular, a discrete-time Fourier trasform (DTFT). A DTFT ca be applied to aalyze the frequecy compoets of a sigal. What we get is a explicit expressio of x() as a superpositio of complex expoetials: x() = 1 2π π π X(e jw )e jw dw where X(e jw ) is the Fourier trasform of x(), a fuctio of the frequecy w: X(e jw ) = x(k)e jwk X(e jw ) is called the spectrum of x(). As a example, cosider the DTFT of the impulse ad its iverse trasform: 6

7 (12) δ() (e jω ) = δ(k)e jωk = δ(0)e jω0 = 1 (e jω ) δ() = 1 π 1 e jω dω 2π π = 1 2π δ() = δ() 2π i.e. the impulse has a uiform spectrum. Similar results also hold for discrete Fourier series. A few remarks: X(e jw ) is ofte referred to as the frequecy domai, while x() as the time domai. X(e jw ) x() is called the iverse Fourier trasform, or the sythesis equatio. x() X(e jw ), the Fourier trasform, is called the aalysis equatio. I order for the Fourier trasform to exist, x() eeds to be either absolutely summable or square summable. The superpositio of x() through its frequecy compoets is achieved through a itegral, which ca be thought of as summig over ifiitely may frequecies. The cotiuous spectrum X(e jw ) is due to the fact that x() is aperiodic (or, periodic with a ifiite period). Usig the iverse Fourier trasform, we ca also determie the impulse respose of a ideal low pass filter: (13) Give H(e jw ) a ideal low pass filter discussed above, we have: h() = 1 π H(e jw )e jw dw 2π π = 1 ω0 1 e jw dw 2π ω 0 1 = 2πj (ejω 0 e jω0 ) = si ω 0 π Although this fuctio is theoretically useful, it is o-causal. Thus a ideal low pass filter caot be realized i the time domai. I practice, approximatios are ofte used. 7

8 Exercise: show that the Fourier trasform is liear, i.e. if x() X(e jw ), y() Y (e jw ), the a x() + b y() a X(e jw ) + b Y (e jw ). Istead of a cotiuous spectrum, a spectrum ca also be discrete (i the sese of a impulse defied earlier), cosider the example of a impulse trai i the frequecy domai: (14) Sice X(e jw ) is periodic, ay impulse at ω 0 will repeat itself every 2π. Cosider X(e jw ) = + 2πδ(w ω 0 2πk). To see what sigal this correspods to i the time domai, applyig the iverse Fourier trasform over [ π, π], ad assume for some t, π 2πt + ω 0 π: x() = 1 π X(e jw )e jw dw 2π π = 1 π δ(w ω 0 2πt)e jw dw 2π π = e j(ω0+2πt) = e jω0 I other words, a impulse trai with uiform height i the frequecy domai correspods to a complex expoetial i the time domai. As a special case, whe ω 0 = 0, the x() is a costat sigal. Q: what is the spectrum of x() = cos ω 0? x() = si ω 0? As a secod applicatio, cosider aother periodic impulse trai that allows the impulses to carry differet weights: (15) Cosider X(e jw ) = + 2πa k δ(w 2πk N ). Applyig the same derivatio for each of the impulse δ(w 2πk N ), we obtai that the time domai sigal is also periodic, ad has period N: x() = N 1 k=0 a k e j 2πk N This represetatio of periodic sigals is also called a Fourier series. For cotiuous sigals x(t), the Fourier series represetatio is: x(t) = a k e jω 0 where ω 0 is the fudametal frequecy of x(t). This idetity states that ay periodic sigals ca be expressed as a superpositio of harmoically related siusoidals, ad is the origi of Fourier aalysis. 8

9 3.3 Fourier trasform ad covolutio Let the impulse respose of a LTI system be h(). The the iput ad output sigals are related through covolutio: y() = + x(k)h( k). I the time domai, this calculatio eeds to sum over a large umber of values. Now let s cosider the frequecy domai ad calculate the spectrum Y (e jw ): Y (e jw ) = = = = y()e jw = = H(e jw ) x(k) x(k)h( k)e jw = = H(e jw ) X(e jw ) h( k)e jw( k) e jwk x(k)e jwk This shows that filterig i the time domai correspod to multiplicatio i the frequecy domai. As a special case, we ca use the impulse to verify the above relatioship: (16) Recall that δ() (e jω ) = 1. Applyig to the covolutio theorem: x() δ() = x() X(e jω ) (e jω ) = X(e jω ) Exercise : usig iverse Fourier trasform to show that Y (e jw ) = H(e jw ) X(e jw ). As the coverse to the covolutio multiplicatio correspodece, we also have the followig result: x() h() = 1 2π X(ejw ) H(e jw ) This ca be verified also by usig the iverse Fourier trasform. 4 Samplig theorem Discrete-time sigals are ofte obtaied from cotiuously varyig sigals by meas of samplig. How fast should oe take samples from the cotiuous sigal x(t) i order to faithfully preserve the cotet of the sigal? The aswer requires we look at the frequecy domai of the cotiuous sigal. A atural 9

10 requiremet for a sampled sigal to reflect the origial cotiuous sigal is that they values must match at iteger times: x[] = x(t), t = T However, x[] ad x(t) are ot defied o the same domai. To facilitate aalysis, we cosider a almost discrete sigal x s (t), which is sampled from x(t) with samplig period T. The precise meaig of samplig is defied by the relatioship betwee x(t) ad x s (t): x s (t) = x(t) δ(t k T ) To cosider the frequecy domai iterpretatio of the samplig operatio, we eed to use the cotiuous Fourier trasform: X(jΩ) = + x(t)e jωt dt The otatio X(jΩ) is used to distiguish it from the discrete-time Fourier trasform X(e jω ), which is periodic i 2π. The discrete-time FT of x() ad the cotiuous trasform of x s (t) (same as x() except for havig 0 o oiteger times) are related as follows: X s (jω) = = = + x s (t)e jωt dt x(t ) e jωt x[] e jωt = X(e j(ωt ) ) Just as the discrete-time FT, cotiuous FT also maps covolutio to multiplicatio, ad vice versa: x(t) h(t) = + x(t) h(t) 1 X(jΩ) H(jΩ) 2π x(s)h(t s) ds X(jΩ) H(jΩ) Applyig it to the idetity x s (t) = x(t) + δ(t k T ), there is: [ X s (jω) = 1 + ] 2π X(jΩ) F T δ(t k T ) 10

11 To determie the cotiuous Fourier trasform of the latter sigal, ote that s(t) = + δ(t k T ) is periodic i T. Usig the Fourier series represetatio of s(t): where a k = 1 T = 1 T s(t) = T/2 T/2 T/2 a k e j 2π T kt s(t)e j(2π/t )kt dt T/2 δ(t)e j(2π/t )kt dt = 1 T Same as i the discrete-time case, each of the harmoic compoet i s(t) correspod to a impulse i the (cotiuous) frequecy domai: Time: e j 2π T kt Frequecy: δ(ω 2π T k) So the cotiuous Fourier trasform of s(t) S(jΩ), is simply aother impulse trai: S(jΩ) = 2π δ(ω 2π T T k) Substitutig this back to X s (jw) = X(jw) S(jw), we have: X s (jω) = 1 2π X(jΩ) 2π T = 1 T X(j(Ω 2π T k)) δ(ω 2π T k) i.e. X s (jω) is a superpositio of may time shifted images of X(jΩ). It follows that i order to avoid frequecy aliasig, the sampled sigal X(jΩ) must be bad-limited by some Ω c, ad 2 Ω c < Ω 0 = 2π/T. Usig the relatioship X s (jω) = X(e jωt ), ad lettig ω = ΩT, we ca also express the samplig theorem as follows: X(e jω ) = 1 T X(j( ω T 2π T k)) The samplig theorem also suggests how to recover the cotiuous sigal x(t) from the sampled versio x(), i a theoretical sese. I order to remove all time-shifted images of X(jΩ) (i.e. X(j(Ω kω 0 )), k 0) i the frequecy domai, we use the aforemetioed ideal low pass filter: { 1, Ωc Ω Ω H(jw) = c 0, Ω c < Ω < π 11

12 Give H(jΩ) also called the ati-aliasig filter, the sampled sigal X s (jω) ad the cotiuous sigal X(jΩ) are related by: T X s (jω) H(jΩ) = X(jΩ) T x s (t) h(t) = x(t) where h(t) is the impulse respose correspodig to the ideal low pass filter, just like i the discrete case: h(t) = si tω c. πt This time domai view suggests that covolvig with a sic fuctio ca be thought as a iterpolatio scheme based o the ideal low pass filter: x(t) = x s (t τ)h(τ) dτ Notice that other more ituitive iterpolatios may be used as a approximatio, for example: x(t) = +t0 t 0 x s (t τ)w(τ) dτ where w(τ) is some type of liear widow fuctio with width t 0 > T. The degree of aliasig for usig such approximatios will the deped o the high frequecy respose (side lobes) of W (jω). Trasportig the samplig/iterpolatio view to the discrete time, similar solutios also apply to the decimatio/iterpolatio problem of discrete sigals. Decimatio i certai applicatios, we wat to lower the samplig rate (for data compressio, etc.) by a iteger umber L. Let the origial sigal be x() (which itself is sampled from some cotiuous sigal x(t)), ad decimated sigal be x (), the: x () = x(l) i.e. we eed to throw away L 1 values for every L samples from x(). Parallel to the samplig theorem, the discrete-time FT of x () ad x() are related i the same way: X (e jω ) = 1 L L 1 k=0 X(e j( ω L 2πk L ) ) The summatio oly eeds to sum over L distict images of X(e jω ) because the latter is already periodic, yet has a differet scalig o both the x ad y axis. Just like samplig from cotiuous sigals, the dowsampled sigal should be bad-limited, ad for this purpose, aother low pass filter, with cutoff frequecy (i radias) determied by 2L ω c < 2π, which follows from the fact that X (e jω ) is obtaied by addig L time-shifted copies of X(e jω ). This leads to the frequecy bad ω c < π/l. The procedure ca be illustrated as follows: x() Low pass filterig by π/l Decimatio by L x () 12

13 Iterpolatio as the iverse of decimatio, we ca also icrease/upsample a discrete sigal to a iteger times (L) of the origial samplig rate. This is typically doe i two steps: first, (L 1) zeros are padded betwee the origial samples, this operatio is cocisely represeted by: x () = k x(k)δ( k L) Takig the discrete FT of both sides, we have: X (e jω ) = x(k)δ( kl)e jω k = x(k)δ( kl)e jω k = x(k)e jωkl = X(e jωl ) k The period of X (e jω ) is 2π/L rather tha 2π. I order to remove the 1,, L 1-th scaled image of X(e jω ), aother low pass filterig is eeded for obtaiig the desired x (), ad the pass bad ω c eeds to satisfy 2ω c < 2π/L. Hece the procedure of iterpolatio ca be illustrated below: x() Zero-paddig by L x () Low pass filterig by π/l x () 5 Relatioship betwee differet types of Fourier trasforms I the discussio of the samplig theorem, we ve ecoutered two kids of Fourier trasforms the cotiuous ad discrete-time Fourier trasform. Their relatioship ca be summarized as follows: Time: x(t) samplig = x s (t) = x(t) k δ(t kt ); i.e. x() Frequecy: X(jΩ) repetitio = X(e jω ) = 1 T X(j( ω T 2π T k)) 13

14 It is helpful to relate this to the Fourier series for cotiuous ad discrete periodic sigals: Time: cotiuous x(t) = x(t + T ) discrete Frequecy: cotiuous a k = 1 T discrete x s (t) = x(t) k T/2 T/2 T N,x(t) x s(t) = a k = 1 N samplig,t N = δ(t k); i.e. x() = x( + N) x(t)e jω 0kt dt, ω 0 = 2π/T, k =,, = N x()e j 2π N k (periodic i N) The Fourier series ad Fourier trasform ca be further related as follows, for both the cotiuous ad discrete sigals: cotiuous Fourier series periodic x(t) = x(t + T ) j(2π/t )kt T x(t) = a k e = x(t) = cotiuous Fourier trasform aperiodic x(t) X(e jω )e jωt dω discrete Fourier series periodic x() = x( + N) x() = N 1 k=0 j(2π/t )kt T a k e discrete-time Fourier trasform aperiodic x(t) = x() = 1 2π π π X(e jω )e jωt dω For the discrete sigals, the aalysis equatios oly iclude a summatio over fiite umber/iterval of frequecy compoets, this is because the frequecy represetatio (a k ad X(e jω )) are periodic (i.e. repeats every N or 2π). The followig observatios are helpful for relatig the time ad frequecy domai: Discrete i time periodicity i frequecy; Periodicity i time Discrete i frequecy. There is yet aother dual relatioship betwee the time ad frequecy domais: Samplig i time repetitio i frequecy; Samplig i frequecy repetitio i time. The former is exemplified by the discussio of the samplig theorem, while the latter will be show through the last example of Fourier trasform discrete Fourier trasform. 14

15 6 Discrete Fourier trasform (DFT) I practice, oe of the Fourier trasforms itroduced above ca be applied to real-world sigals for the followig reasos: first, we ofte do ot kow whether a sigal is periodic or aperiodic (therefore havig discrete or cotiuous spectrum). Secod, eve if we kow a sigal is periodic, it is ot trivial to fid it s period; Third, real-world sigals are always fiite i duratio. I geeral, with computers, we have to implemet a discretized versio of the discretetime Fourier trasform o a fiite-legth sigal that ca be computed efficietly. Therefore it is importat to kow the cosequeces of such a operatio. Amog the three problems metioed above, the third oe is the most fudametal. The basic solutio to this is to ifiitely exted the fiite sigal so that it will become periodic. More precisely, give a fiite-legth sequece x() with duratio M, we cosider aother sequece: x() = k x( + kn) = x( mod N) where N is chose such that N M. If N > M, the extra zeros ca be padded to the ed of x(). Compared to x(), x() is idetical o = 0,, M 1. But differet from x(), x() is periodic, which meas that it has a discrete spectrum. (therefore ca be implemeted o a computer!) The aalysis/sytheis equatios are: ã k = 1 N x() = N 1 =0 N 1 k=0 x()e j(2π/n)k ã k e j(2π/n)k Usig the fact that x() ad x() are idetical betwee = 0 ad N 1, this leads to the defiitio of discrete Fourier trasform pair: X[k] = 1 N x() = N 1 x()e j(2π/n)k =0 N 1 X[k]e j(2π/n)k k=0 Now compare X[k] to the true spectrum of x() X(e jω ). Usig the fact that x() oly has fiite legth (icludig the padded zeros): X(e jω ) = N 1 =0 x()e jω 15

16 Compare this with the DFT X[k], we obtai the relatioship (for a fixed k): X[k] = X(e jω ) ω=(2π/n) k If we itroduce a circular impulse trai, we may also write: X[k] = X(e jω ) π π δ(ω 2π k) dω N This shows that we have the dual result to the samplig theorem: samplig i frequecy results i repetitio i time. As a cosequece, it follows that there must be N > M, i.e. the DFT poits must be loger tha the duratio of the sequece, so that aliasig will ot occur i the time domai. 16

17 7 Discrete cosie trasform I speech applicatios, the most commoly used iformatio is the magitude of the discrete Fourier trasform. This is certaily a loss of iformatio, sice we ve completely igored the phases of the harmoic compoets. A alterative to Fourier trasform is lookig for trasforms i which the harmoic coefficiets are all real. A example of this type of real trasforms is the discrete cosie trasform. As metioed previously, real sigals have cojugate-symmetric fourier trasforms. It is worth metioig that coverse is also true: symmetric sigals have real fourier trasforms (ca be easily verified through the defiitio). However, real-world, causal sigals are ot symmetric. Thus we take a similar strategy i defiig the discrete Fourier trasform extedig the sigal to make it periodic. But for the curret purpose, simply repeats the sigal will ot guaratee symmetry. I particular, give a sigal x(), we cosider aother sigal x() as follows: { x(), 0 N; x() = x( ), N < 0. x( + 2N) = x() I other words, we first flip the sigal to < 0, ad repeat the symmetric sigal every 2N samples. The DCT is (roughly) defied as the discrete Fourier series of this periodic sequece. Therefore, the crucial differece betwee DFT ad DCT is the way the origial sigal is exteded. I DCT, the exteded sigal has period 2N, while i DFT, the period is N (therefore DCT is ot just the real part of DFT!). Differet variats of DCT are mostly due to differet ways of extedig the sequece at the boudary poits, i.e. = ±N. For example, the DCT-2 is defied as: X DCT 2 [k] = N 1 =0 ( ) πk(2 + 1) x() cos 2N This replaces the complex orthoormal basis e jωk, k = 0, 1, with real basis cos ωk. This set of basis is still orthoormal, but is completely real-valued. Similarly it is also possible to use si ωk as a real basis, leadig to the so-called discrete sie trasform. 17

18 8 Z-trasform ad the trasform aalysis of differece equatios So far we have avoided mathematical rigor i the discussio of Fourier trasforms. Some fuctios may ot be itegrable, some series may ot coverge, ad summatio/itegral may ot be iterchageable. For example, if a system is ustable, the its impulse respose may ot have a fourier trasform. Moreover, by writig z = e jω, we captured the fact that the complex variables all lie o the uit circle. A atural geeralizatio is to cosider the whole complex Z plae rather tha the uit circle, ad thus we have the Z-trasform: X(z) = x(k)z k Although this is simply a chage of variable from the DTFT, its domai is the whole complex plae. A atural questio is for a give x(), what are choices of z that make X(z) well-defied (coverge)? This issue is called the Regio of Covergece(ROC). We state a few importat facts relevat to ROC without provig them: If h() is causal (right-sided), the the ROC of H(z) have the followig form: {z : z C}. If h() is stable (i.e. h() < + ), the the ROC of H(z) must iclude the uit circle. A particular class of system has the so-called ratioal Z-trasforms, which meas that H(z) looks like the follows: H(z) = B(z) A(z) where B(z), A(z) are both polyomials i z with real coefficiets. Amog these systems, a particular sub-class is the oes described by liear differece equatios. Usig the defiitio give before i the otes, these equatios look like: N M a k y( k) = b l x( l) k=0 What if we apply the Z-trasform to both sides of the equatio? A importat property (which also holds for the Fourier trasform) is the time-delay property: x () = x( N) X (z) = l=0 = z N 18 x(k N)z k x(k)z k = z N X(z)

19 Applyig this to the LDE, we have: N a k z k Y (z) = k=0 H(z) = Y (z) X(z) = M b l z l X(z) l=0 M b l z l l=0 = N a k z k k=0 B(z) A(z) Sice we will be mostly dealig with causal systems, it s more coveiet to use the polyomials of z 1 rather tha z. The roots of A(z) are called poles, sice the value H(z) is large ear these roots; ad roots of B(z) are called zeros. Roots ad zeros are geerally complex-valued, ad may lie iside or outside the uit circle. Without further simplificatio of the fractio B(z)/A(z), some poles ad zeros may cacel, ad there may be duplicate roots. By the fudametal theorem of algebra, the complex roots form cojugate pairs. Therefore they should be symmetric with respect to the real axis. All the algebraic properties of the Fourier trasform carry over the Z-trasform. Parallel to the iverse Fourier trasform, we ca also defie the iverse Z- trasform based o the Cauchy itegral theorem. Details of this ca be foud i stadard refereces ad will be omitted here. Poles ad zeros are importat for determiig the frequecy respose of a filter, sice we are simply evaluatig H(z) = B(z)/A(z) o the uit circle e jω, ω [0, 2π). The followig facts are worth kowig: FIR filter oly has zeros, while IIR filters have poles (ad possibly zeros as well). For stable ad causal systems (e.g. vocal tract), the poles of H(z) must lie iside the uit circle. These properties ca be verified by followig the defiitios. A few examples follow: (17) A high-pass filter: H(z) = 1 a z 1, a > 0 This filter has a zero at z = a, ad is based o the FIR system y() = x() a x( 1). (18) A low-pass filter: H(z) = 1 1 a z 1 This filter has a pole at z = a, ad is based o the IIR system y() a y( 1) = x(). I order for H(z) to be stable, the pole z = a must lie iside the uit circle, so that the ROC (goig outwards from the pole) will iclude z = 1. 19

20 (19) A secod-order resoator: H(z) = 1 1 a 1 z 1 a 2 z 2 This filter has two poles i the geeral form of: where α, β satisfy: (20) A all-pole system: z = e α±jβ z 1 + z 2 = e α 2 cos β = a 1 z = z 1 z 2 = e α = a 2 H(z) = K H k (z) k=1 where H k (z) are secod-order resoators as described above. Each H k (z) cotributes two cojugate complex roots z = e α k±jβ k, where α k /2 is referred to as the badwidth, ad the β k is called the resoace frequecy of the k-th filter. It may be see that ω = ±β correspods to the maximum magitude respose of H(z), while whe α 0 from below, the chage i H(z) becomes more dramatic. The all-pole system is the basis of a large umber of applicatios, icludig the liear predictio methods to be discussed later. 9 The all-pole physical model ad the wave equatio I this sectio, we wat to demostrate how a physical model ca be used to derive the resoace properties of a uiform vocal tract, thereby justifyig the use of the all-pole model, which ca be see as a abstractio of the true uderlyig physical model. Cosider a uiform tube, with a closed ed (the glottis) ad a ope ed (the lips). Such a tube is a reasoable model of the vowel [@], as the togue is restig i a atural positio ad does ot create ay costrictios i the vocal tract. Our goal is to derive a relatioship betwee the volume velocity (defied as particle velocity area) of the air at the lips ad at the glottis. Usig Fourier trasform, these two quatities (deoted as u(0, t) ad u(l, t) ca be expressed as a sum of its frequecy compoets: u(0, t) = u(l, t) = U g (Ω)e jωt dω U l (Ω)e jωt dω 20

21 For example, if u(0, t) is simply a siusoidal, the it ca be writte as u(0, t) = U g (Ω 0 )e jω0t. Parallel to the Fourier trasform of the LTI system, the characteristics of the vocal tract ca be expressed through the followig ratio: T (Ω) = U l(ω) U g (Ω) T (Ω) is called the trasfer fuctio of the vocal tract. The resoace frequecies are the poles of the trasfer fuctio. We start from Newto s secod law. Give the assumptio of icompressible fluid, the pressure differece ( p(x, t)/ x) dx accouts for the acceleratio of the air at the ifiitesimal volume dx dy dz. Multiplyig the pressure differece with the area dy dz o the left, ad the desity ad the volume o the right, we have: p(x, t) v(x, t) (dxdy) = ρ(dxdydz) x t Usig the volume velocity u(x, t) = Av(x, t), we have: p(x, t) = ρ u(x, t) x A t I order to solve for p(x, t) ad v(x, t), we eed aother equatio. This equatio ca be derived from the gas law: u(x, t) x = A p(x, t) ρc2 t By direct substitutio, it ca be verified that the solutio has the followig form: u(x, t) = u + (t x c ) u (t + x c ) p(x, t) = ρc [u + (t x A c ) + u (t + x ] c ) Here u + ad u have the physical iterpretatios of forward ad backward travelig waves. They are amed such because u + (t x/c) is u + (t) shifted to the right by x/c; while u (t+x/c) is u (t) shifted to the left by x/c. Ituitively, whe the forward-travelig wave reaches the lips, part of the eergy is reflected back to the vocal tract. Similarly, the backward travelig wave also reflects whe it reaches the glottis. The iterferece of these two waves leads to resoace. Theoretically, without loss of eergy, such a resoator could resoatig forever with the right choice of frequecy. Now to determie the frequecy respose of the tube with legth = L, we apply the siusoidal as the glottal excitatio: u(0, t) = U g (Ω)e jωt Roughly this says the volume velocity at the glottis is drive by a pisto whose movemet produces a siusoidal. Sice these differetial equatios are liear, 21

22 the forward ad backward travelig waves have the same expoetial form: u + (t x c ) = k+ e jω(t x c ) u (t + x c ) = k e jω(t+ x c ) Usig a secod boudary coditio p(l, t) = 0 (e.g. o pressure drop at the lips), we ca solve for k + ad k from the followig equatios: The result is: U g (Ω)e jωt = k + e jωt k e jωt ρc ( k + e jω(t L c ) + k e jω(t+ L )) c = 0 A k = U g(ω)e jωl/c e jωl/c + e jωl/c k + = U g (Ω)e jωl/c e jωl/c + e jωl/c Puttig them back to the expressio for u(x, t): u(x, t) = cos [ Ω( x c L c )] cos ΩL U g (Ω)e jωt c Notice this expressio of u(x, t) o loger has a forward ad backward travelig compoet, ad the magitude oly depeds o the distace from the glottis x. The decouplig of x ad t i the wave equatio correspods to the stadig wave pheomeo: cos[ω( x c L c )] cos ΩL c provides a evelope of the wave over the legth of the vocal tract, ad the shape of the wave appears time-ivariat. Lettig x = L (which correspods to the maximum volume velocity at the lips), the trasfer fuctio at the lip is the follows: U l (Ω) U g (Ω) = 1 cos ΩL c The poles of this trasfer fuctio ca be directly derived as: ΩL c = π, = 0, 1, 2, 2 where Ω is the agular frequecy. Ituitively, such choices of Ω will fit a ode of the stadig wave at the closed ed of the tube, with a ati-ode at the ope ed. The resoace frequecy f is: f = Ω 2π (2 + 1)c =, = 0, 1, 2, 4L For example, usig the speed of soud = 340m/sec, vocal tract legth = 17cm, the formats of [@] is 500Hz, 1500Hz, 2500Hz, etc. 22

23 The above aalysis suggests a ifiitely large frequecy respose at the resoace, which correspods to a zero badwidth. However, i reality, losses due to the o-rigid walls, viscosity of air, radiatio from lips all cotribute to the o-zero badwidth of the formats. Agai, physical models are available for derivig the badwidths. The aalysis of the wave equatio based o forward ad backward travelig waves ca be also applied to a cocateatio of tubes, thereby providig the basis for modelig a large umber of vocal tract cofiguratios. Details of such modelig techiques ca be foud i (Steves, 1998) ad (Rabier ad Schaefer, 1978). 23

24 10 Short-time processig The example of discrete Fourier trasform provides importat hits about the basic approach to speech processig. Although speech is ot a statioary sigal, our basic tools assume statioarity. Moreover, computatio usig fiite amout of data is made possible uder the assumptio of periodicity. I speech processig, we will be takig may sapshot of the speech sigal, ad these sapshots will be called frames. Frames are obtaied by multiplyig the origial sigal with a shiftig widow fuctio. { 1, 0 N 1 Rectagular widow: w() = 0, otherwise. The effect of the rectagular widow is just limitig whatever operatios to a chuk of sigal with legth N. Istead of havig a sharp cutoff, there are also a class of widow fuctios that try to smoothly taper the sigal at the edge of the widow: { α (1 α) cos 2π w() = N 1, 0 N 1 0, otherwise. Hammig widow: α = 0.54 Haig widow: α = 0.5 The followig widow is used i the so-called Gabor trasform: Gaussia widow: x() = 1 ( 2πσ 2 e 0 )2 2σ 2 I practice, the widow fuctio shifts over time, ad the rate of shiftig is called the frame rate. Give a discrete sigal, the highest frame rate is the same as the samplig rate, thus resultig i desely overlappig frames. I practice, the frame rate is ofte chose to produce partially overlap frames, ad the commo practice is to let the overlap be half of the widow legth. The geeral defiitio for a widowed sequece is the followig: y (m) = x(m) w (m) where idicates the time-shift of the widow. This ca be put ito the more coveiet otatio: y (m) = x(m) w( m) Here w( m) is w(m) flipped across the Y-axis ad the shifted by. Usig this otatio, we ca defie the followig short-time properties of a sigal: Short-time sigal eergy: E = m (x(m)w( m)) 2 24

25 Short-time zero crossig rate: ZR = m [sg x(m) sg x(m 1)] w( m) Short-time autocorrelatio: R (k) = m x(m)x(m + k)w( m)w( m k) I theoretical discussios, usig the above otatio for short-time processig has the advatage that may of the sums have the covolutio iterpretatio, ad widowig ca be see as a filterig process. For example, the short-time eergy ca be see as x 2 () the istataeous eergy filtered by w 2 (). I practice, the summatio is always performed withi a fiite widow. For example, if the rectagular widow of legth N is used (i.e. the sigal is selected from x() to x( + N 1), the the short-time autocorrelatio is defied as: R (k) = N k 1 m=0 x( + m) x( + m + k) where k rages from N to N. Whe we defie x() to be zero outside the widow, it ca be see that R (k) is a eve fuctio of k. The operatio of widowig is essetial for a umber of speech processig tasks, sice speech sigals are time-varyig, ad the iterest is always o locatig the evets of speech productio i the sigal. However, as ca be see from the defiitio of the short-time processig, the effect of widow legth is obvious: Short widows ted to lead to bumpy estimates. Log widows ted to miss trasiet chages i the sigal, e.g. release of a stop. To aswer the questio what widow to use, we will eed to cosider the effect of widowig i the frequecy domai. 11 Short-time Fourier trasform The role of widow sequeces is to select a portio of the sigal, ad effectively reducig the computatio to a fiite-legth sequece. I the frequecy domai, the iterpretatio of this operatio is give by the followig correspodece: x() w() 1 2π X(ejω ) W (e jω ) Here W (e jω ) is the DTFT of the widow sequece, ad is iterpreted as circular covolutio, sice both X(e jω ) ad W (e jω ) are periodic i 2π. The followig observatios follow: 25

26 The ideal widow that preserves X(e jω ) exactly, would have DTFT W (e jω ) = 2πδ(ω). Iverse trasform shows that i this case: w() = 1 π 2πδ(ω)e jω dω = 1 2π π i.e. w() is a ifiitely log costat sequece. I other words, we are simply failig to locate the sigal precisely i time. The ideal widow that has the best time resolutio, will be the delta fuctio δ(). But as see i previous examples, δ() has a uiform spectrum, therefore resultig the worst frequecy resolutio amog all possible widows. The rectagular widow, which has a fiite legth. The Fourier trasform of such a widow, however, spreads over a large frequecy rage. Moreover, it is easy to verify that the shorter the rectagular widow, the larger the spread. To see this, compute the DFT of the rectagular widow with legth N: W (e jω ) = = = N 1 =0 x()e jω e jω = 1 e jωn 1 e jω Whe we covolve such a W (e jω ) with the origial X(e jω ), we ca fail to locate the sigal i frequecy. Imagie, we observe a chuk of the siusoidal. I order to make sure it is really a siusoidal, we would have to icrease our observatio widow. This will lead to the arrowig of the spectrum of the widowed sequece because the siusoidal evetually has zero spread i frequecy. However, if we limit our observatio to oe period, the such a widowed sequece (o loger a periodic oe), will have o-zero side lobes spreadig over the higher frequecies i the spectrum. This dilemma i selectig the legth of a aalysis widow is a aspect of the geeral ucertaity priciple, which roughly states that a arrow sigal has a wide spectrum, ad a wide sigal has a arrow spectrum. I order to miimize the distortio due to the high frequecy compoet of W (e jω ), a popular choice is to use the more smooth cosie widows (e.g. Hammig). From the perspective of the DFT implemetatio, usig a taperig widow also helps esure that the exteded sequece is approximately cotiuous, thereby reducig spurious high frequecy compoets i DFT. 26

27 12 Spectrograms The short-time Fourier trasform of the sigal multiplied with a series of shiftig widows is called a spectrogram. Usig a otatio similar to the short-time eergy, the spectrogram is defied as: X (e jω ) = m= x(m)w( m)e jωm Here represets the time idex of the widow, ad X (e jω ) depeds o both time ad frequecy. I implemetatio, the widow is shifted at M samples a time, while ω is discretized as i DFT: ω = (2π/N)k, k = 0, 1,, N 1. Hece the implemetatio uses: X l [k] = m= x(m)w(l M m)e j 2π N km, k = 0, 1,, N 1 I practice, the summatio almost always has a fiite rage sice a fiite widow is ofte used. The discrete idex l ad k idicate that spectrogram ivolves samplig i both time ad frequecy. What does ot appear i the defiitio of the spectrogram is the widow legth. Here the same priciple of time-frequecy resolutio tradeoff applies: Short time widow offers better time resolutio, but blurs the spectral profile of speech souds (recall that shorter widows have wider Fourier trasforms) Log time widow offers better frequecy resolutio (recall that loger widows have arrower spectrum), but the statioarity assumptio fails to match the time-varyig sigal. I practice, the widow legth is ofte set to a few fudametal periods, which i tur depeds o the pitch of the speaker. Whe the widow legth w() is log eough (which implies a arrow W (e jω )) so that the idividual harmoic compoets ca be see from the widowed sequece W (e jω ) X(e jω ), the spectrogram is called arrow bad. Otherwise, it s called a wide bad spectrogram. 13 Liear predictio methods 13.1 Geeral formulatio Short-time Fourier trasform provides a frequecy domai characterizatio of the origial speech sigal. However, the STFT represetatio is still very high dimesioal, ad cotais iformatio (radiatio characteristics, etc.) that is ot essetial for idetifyig the cotet of speech. Oe approach is seekig a parametric model of the spectral property of the sigal, with the umber of 27

28 parameters much smaller tha the origial sigal, ad which provides a more smooth spectrum tha the origial STFT. Recall the all-pole model: p a k y( k) = b x() k=0 To remove redudacy i parameters, we let a 0 x() = u(), ad cosiders the followig system: = 1, ad the iput sigal y() = p a k y( k) + Gu() k=1 This equatio says that a sample at ay time is a liear combiatio of the past samples, hece the ame liear predictio. I frequecy domai, this model has the followig system fuctio: H(z) = G 1 p k=1 a kz k The liear predictio method addresses the problem of fittig the parameters {a 1,, a k, G} to a give speech sigal s(). The criterio for fittig this model is the so-called least-square procedure: arg mi [ s() p a k s( k) The sum of squares term beig miimized is the eergy of the followig sigal: e() = s() k=1 p a k s( k) k=1 e() is called predictio error. Thikig of e() as s() filtered by aother system, we have: E(z) p S(z) = 1 a k z k which shows that this is the iverse system of the all-pole system. Sice the true iput sigal x() ca be approximated by a impulse trai with small eergy, miimizig the eergy of e() is a reasoable assumptio. Because of the similarity betwee this formulatio ad liear regressio (roughly, regressig the sequece to itself), it is also called a autoregressive model. The ext step is to determie the value of a k. Differetiatig the eergy k=1 ] 2 28

29 with respect to a j, we have: s 2 () 2s() a j [ 2 p s() a k s( k)] = 0 a j k=1 ( p p ) 2 a k s( k) + a k s( k) = 0 k=1 k=1 [ 0 2s()a j s( j) + 2a j s( j) Iterchagig the summatio sigs, we have: ] p a k s( k) k=1 = 0 p a j s()s( j) + a j a k s( j)s( k) = 0 k=1 p a k s( j)s( k) = k=1 s()s( j) Now let j rages from 1,, p, the we have a set of p equatios with p ukows: p a k s( 1)s( k) = p a k s( p)s( k) = k=1 k=1 s()s( 1) s()s( p) They form a liear system, ad ca be compactly writte i the matrix form: a 1 a 2 s( 1)s( 1) s( 1)s( 2) s( 1)s( p) s( 2)s( 1) s( 2)s( 2) s( 2)s( p) = a p s( p)s( 1) s( p)s( 2) s( p)s( p) A direct approach is simply solvig this liear system for the ukow a k, ad much work has bee doe to facilitate computatio by makig use of the structure of the p p matrix. (Rabier ad Schaefer, 1978) provides most of the details Short-time liear predictio Same as the previous approach, the short-time liear predictio adds timedepedece to the widowed sigal s (m), which meas we eeds to solve the liear system for each widowed sigal s (m). By assumig that s (m) is zero s()s( 1) s()s( 2) s()s( p) 29

30 outside the widow, the liear system formulated above ca be writte i terms of short-time autocorrelatio, sice: s (m i)s (m j) = R (i j) = R (j i) m Here we use the property of R beig a eve fuctio. Substitutig R i the liear system, we have: a 1 R (0) R (1) R (p 1) R (1) a 2 R (1) R (0) R (p 2) = R (2) R (p) R (p 1) R (0) R (p) a p The p p matrix is ot oly symmetric, but oly has p distict elemets that shift i a circular fashio. This matrix is called a toeplitz matrix, which leads to fast solutio of the liear system. Alteratively, if we assume the sigal is o-zero outside the aalysis widow: 0 m N 1, the N 1 m=0 s (m i)s (m j) is o loger the short-time autocorrelatio fuctio, sice it eeds to use values of s ( 1),, which lie outside of the widow. This approach leads to the so-called covariace method for solvig liear predictio equatios. 30

31 14 Homomorphic processig ad cepstrum A motivatio for doig liear filterig is the issue of sigal separatio. Suppose i time domai, two sigals are additive: z() = x() + y() The they are also additive i the frequecy domai: Z(e jω ) = X(e jω ) + Y (e jω ) Two sigals that are overlappig i time ca be separate i frequecy, ad we ca choose filters with the desired frequecy respose to separate them. Homomorphic processig is used to address a differet kid of sigal separatio problem: what if the two sigals are covolutioal i time? I this case, liear filterig will ot work, sice: x() y() = X(e jω ) Y (e jω ) This problem appears i situatios where we are oly iterested i the cotet of the speech X(e jω ), ot the voice Y (e jω ). Refereces Rabier, L. R. ad R. W. Schaefer Digital Processig of Speech Sigals. Pretice Hall, Eglewood Cliffs, NJ. Steves, Keeth Acoustic Phoetics. MIT Press, Cambridge, MA. 31