Lecture 3: Divide and Conquer: Fast Fourier Transform

Transcription

1 Lecture 3: Divide ad Coquer: Fast Fourier Trasform Polyomial Operatios vs. Represetatios Divide ad Coquer Algorithm Collapsig Samples / Roots of Uity FFT, IFFT, ad Polyomial Multiplicatio Polyomial operatios ad represetatio A polyomial A(x) ca be writte i the followig forms: 1 A(x) = a 0 + a 1 x + a x + + a 1 x 1 = a k x k k=0 = (a 0,a 1,a,...,a 1 ) (coefficiet vector) The degree of A is 1. Operatios o polyomials There are three primary operatios for polyomials. 1. Evaluatio: Give a polyomial A(x) ad a umber x 0, compute A(x 0 ). This ca be doe i O() time usig O() arithmetic operatios via Horer s rule. Horer s Rule: A(x) = a 0 +x(a 1 + x(a + x(a 1 ) )). At each step, a sum is evaluated, the multiplied by x, before begiig the ext step. Thus O() multiplicatios ad O() additios are required.. Additio: Give two polyomials A(x) ad B(x), compute C(x) = A(x) + B(x) ( x). This takes O() time usig basic arithmetic, because c k = a k + b k. 1

2 3. Multiplicatio: Give two polyomials A(x) ad B(x), compute C(x) = k A(x) B(x)( x). The c k = j=0 a j b k j for 0 k ( 1), because the degree of the resultig polyomial is twice that of A or B. This multiplicatio is the equivalet to a covolutio of the vectors A ad reverse(b). The covolutio is the ier product of all relative shifts, a operatio also useful for smoothig etc. i digital sigal processig. Naive polyomial multiplicatio takes O( ). O( lg 3 )oreve O( 1+ε )( ε > 0) is possible via Strasse-like divide-adcoquer tricks. Today, we will compute the product i O( lg ) time viafastfourier Trasform! Represetatios of polyomials First, cosider the differet represetatios of polyomials, ad the time ecessary to complete operatios based o the represetatio. There are 3 mai represetatios to cosider. 1. Coefficiet vector with a moomial basis. Roots ad a scale term A(x) =(x r 0 ) (x r 1 ) (x r 1 ) c However, it is impossible to fid exact roots with oly basic arithmetic operatios ad kth root operatios. Furthermore, additio is extremely hard with this represetatio, or eve impossible. Multiplicatio simply requires roots to be cocateated, ad evaluatio ca be completed i O(). 3. Samples: (x 0,y 0 ), (x 1,y 1 ),..., (x 1,y 1 )with A(x i )= y i ( i) ad each x i is distict. These samples uiquely determie a degree 1 polyomial A, accordig to the Lagrage ad Fudametal Theorem of Algebra. Additio ad multiplicatio ca be computed by addig ad multiplyig the y i terms, assumig that the x i s match. However, evaluatio requires iterpolatio. The rutimes for the represetatios ad the operatios is described i the table below, with algorithms for the operatios versus the represetatios.

3 Algorithms vs. Represetatios Coefficiets Roots Samples Evaluatio O() O() O( ) Additio O() O() Multiplicatio O( ) O() O() We combie the best of each represetatio by covertig betwee coefficiets ad samples i O( lg ) time. How? Cosider the polyomial i matrix form. V A = 1 x 0 x 0 x 1 0 a 0 y0 1 1 x1 x 1 x 1 a1 y 1 1 x x x 1 a = y x 1 x 1 x a y 1 where V is the Vadermode matrix with etries v jk = x k j. The we ca covert betwee coefficiets ad samples usig the matrix vector product V A, which is equivalet to evaluatio. This takes O( ). Similarly, we ca samples to coefficiets by solvig V \Y (i MATLAB otatio). This takes O( 3 ) via Gaussia elimiatio, or O( ) to compute A = V 1 Y,ifV 1 is precomputed. To do better tha Θ( ) whe covertig betwee coefficiets ad samples, ad vice versa, we eed to choose special values for x 0,x 1,...,x 1. Thusfar, wehave oly made the assumptio that the x i values are distict. Divide ad Coquer Algorithm We ca formulate polyomial multiplicatio as a divide ad coquer algorithm with the followig steps for a polyomial A(x) x X. 1. Divide the polyomial A ito its eve ad odd coefficiets: 1 11 A eve (x) = a k x = (a 0,a,a 4,...) k=0 l 1 J k Aodd(x) = a k+1 x = (a 1,a 3,a 5,...) k=0 k 3

4 . Recursively coquer A eve (y) for y X ad A odd (y) for y X,where X = {x x X}. 3. Combie the terms. A(x) = A eve (x )+ x A odd (x )for x X. However, the recurreces for this algorithm is ( ) T (, X ) = T, X + O( + X ) = O( ) which is o better tha before. We ca do better if X is collapsig: either X = 1 (base case), or X = X ad X is (recursively) collapsig. The the recurrece is of the form ( ) T () = T + O() = O( lg ). Roots of Uity Collapsig sets ca be costructed via square roots. Each of the followig collapsig sets is computig by takig all square roots of the previous set. 1. {1}. {1, 1} 3. {1, 1,i, i} 4. {1, 1, ± (1 + i), ± ( 1+ i)}, which lie o a uit circle We ca repeat this process ad make our set larger ad larger by fidig more ad more poits o this circle. These poits are called the th roots of uity. Formally, the th roots of uity are x s such that x = 1. These poits are uiformly spaced aroud the uit circle i the complex plae (icludig 1). These poits are of the form (cos θ, si θ) =cos θ+i si θ = e iθ by Euler s Formula, for θ = 0, 1 τ, τ,..., 1 τ (where τ =π). The th roots of uity where = form a collapsig set, because (e iθ ) = e i(θ) = i(θ mod τ ) e. Therefore the eve th roots of uity are equivalet to the d roots of uity. 4

5 FFT, IFFT, ad Polyomial Multiplicatio We ca take advatage of the th roots of uity to improve the rutime of our polyomial multiplicatio algorithm. The basis for the algorithm is called the Discrete Fourier Trasform (DFT). The DFT allows the trasformatio betwee coefficiets ad samples, computig A A = V A for x k = e iτk/ where =, where A is the set of coefficiets ad A 1 iτjk/ is the resultig samples. The idividual terms a = e a j. Fast Fourier Trasform (FFT) j j=0 The FFT algorithm is a O( lg ) divide ad coquer algorithm for DFT, used by Gauss circa 1805, ad popularized by Cooley ad Turkey ad Gauss used the algorithm to determie periodic asteroid orbits, while Cooley ad Turkey used it to detect Soviet uclear tests from offshore readigs. A practical implemetatio of FFT is FFTW, which was described by Frigo ad Johso at MIT. The algorithm is ofte implemeted directly i hardware, for fixed. Iverse Discrete Fourier Trasform The Iverse Discrete Fourier Trasform is a algorithm to retur the coefficiets of a polyomial from the multiplied samples. The trasformatio is of the form A V 1 A = A. I order to compute this, we eed to fid V 1, which i fact has a very ice structure. Claim 1. V 1 1 = V, where V is the complex cojugate of V. 1 1 Recall the complex cojugate of p + qi is p qi. 5

6 Proof. We claim that P = V V = I: p jk = (row j of V ) (col. k of V ) 1 = e ijτm/ e ikτm/ 1 = e e 1 = e ijτm/ ikτm/ i(j k)τm/ Now if j = k, p jk = 1 =. Otherwise it forms a geometric series. 1 p jk == (e i(j k)τ/ ) m (e iτ(j k)/ ) 1 = e iτ (j k)/ 1 =0 because e iτ =1. Thus V 1 1 = V,because V V = I. This claim says that the Iverse Discrete Fourier Trasform is equivalet to the ikτ/ Discrete Fourier Trasform, but chagig x k from e to its complex cojugate e ikτ/, ad dividig the resultig vector by. The algorithm for IFFT is aalogous to that for FFT, ad the result is a O( lg ) algorithm for IDFT. Fast Polyomial Multiplicatio I order to compute the product of two polyomials A ad B, we ca perform the followig steps. 1. Compute A = FFT(A) ad B = FFT(B), which coverts both A ad B from coefficiet vectors to a sample represetatio.. Compute C = A B i sample represetatio i liear time by calculatig C = A B ( k). k k k 3. Compute C = IFFT(C ), which is a vector represetatio of our fial solutio. 6

7 Applicatios Fourier (frequecy) space may applicatios. The polyomial A = FFT(A) iscomplex, ad the amplitude a k represets the amplitude of the frequecy-k sigal, while arg(a k ) (the agle of the D vector) represets the phase shift of that sigal. For example, this perspective is particularly useful for audio processig, as used by Adobe Auditio, Audacity, etc.: High-pass filters zero out high frequecies Low-pass filters zero out low frequecies Pitch shifts shift the frequecy vector Used i MP3 compressio, etc. 7

8 MIT OpeCourseWare J / J Desig ad Aalysis of Algorithms Sprig 015 For iformatio about citig these materials or our Terms of Use, visit: