MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 2/42 Operatios o polyomials MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 4/42 Efficiet Patter Matchig (MI-EVY) 4. DFT, FFT ad Patter Matchig We are particularly iterested i two operatios: Polyomial additio Polyomial multiplicatio Ja Holub Departmet of Theoretical Computer Sciece Faculty of Iformatio Techology Czech Techical Uiversity i Prague c Ja Holub, 2011 Evropský sociálí fod. Praha & EU: Ivestujeme do vaší budoucosti MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 1/42 Polyomials MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 3/42 Polyomial additio Defiitio A polyomial of degree 1 i the variable x over a algebraic field F represets a fuctio A(x) as a formal sum: A(x) = 1 a i x i where a i, 0 i < are the coefficiets of the polyomial. A(x) = 7x 3 + 4x 2 + x + 3 Let A(x) ad B(x) be two polyomials of degree 1: The sum of polyomials A(x) ad B(x) is also a polyomial C(x) of degree 1: C(x) = (a i + b i )x i 2 1 Let A(x) = 3x 3 + 2x 2 + x + 1 ad B(x) = 2x 3 + 3x 2 + 4x + 3. The sum A(x) + B(x) is C(x) = 5x 3 + 5x 2 + 5x + 4.
Polyomial multiplicatio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 6/42 Coefficiet represetatio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 8/42 Let A(x) ad B(x) be two polyomials of degree 1: The product of polyomials A(x) ad B(x) is a polyomial C(x) of degree 2 2. C(x) = c i x i where c i = i a j b i j Let A(x) = 3x 3 + 2x 2 + x + 1 ad B(x) = 2x 3 + 3x 2 + 4x + 3. The product A(x)B(x) is C(x) = 6x 6 + 13x 5 + 20x 4 + 22x 3 + 13x 2 + 7x + 3. j=0 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 5/42 Represetatios of polyomials Let A(x) be a polyomial of degree 1 havig the form: Its coefficiet represetatio is the vector of coefficiets α = (a 0, a 1,..., a 1 ) Let A(x) = 3x 3 + 2x 2 + x + 1. Its coefficiet represetatio is the vector α = (1, 1, 2, 3). MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 7/42 Coefficiet represetatio - Evaluatio Coefficiet represetatio Poit-value represetatio Give the coefficiet represetatio α = (a 0, a 1,..., a 1 ) of a polyomial of degree 1, evaluatio ca be doe i Θ() time usig Horer s rule: A(x 0 ) = a 0 + x 0 (a 1 + x 0 (a 2 + + x 0 (a 2 + x 0 (a 1 )) ))
Coefficiet represetatio - Additio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 10/42 Coefficiet represetatio - Summary MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 12/42 Give the coefficiet represetatios of polyomials α = (a 0, a 1,..., a 1 ) β = (b 0, b 1,..., b 1 ) Evaluatio i Θ() Additio i Θ() Multiplicatio i Θ( 2 ) of degree 1, additio ca be doe i Θ() time by producig the coefficiet vector γ = (c 0, c 1,..., c 1 ) where c i = a i + b i for i = 0, 1,..., 1. MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 9/42 Coefficiet represetatio - Multiplicatio Give the coefficiet represetatios α = (a 0, a 1,..., a 1 ), β = (b 0, b 1,..., b 1 ) of polyomials of degree 1, multiplicatio ca be doe i Θ( 2 ) time by producig the coefficiet vector γ = (c 0, c 1,..., c 2 2 ), c i = j a j b i j The resultig vector γ is also called the covolutio of vectors α ad β, deoted γ = α β. j=0 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 11/42 Poit-value represetatio Let A(x) be a polyomial of degree 1 havig the form: Its poit-value represetatio is a set of poit-value pairs {(x 0, y 0 ), (x 1, y 1 ),..., (x 1, y 1 )} where y i = A(x i ) for i = 0, 1,..., 1. Let A(x) = 3x 3 + 2x 2 + x + 1. Its poit-value represetatio ca be the vector {(0, 1), (1, 7), (2, 35), (3, 103)}
Poit-value represetatio Cot. MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 14/42 Poit-value represetatio - Additio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 16/42 A polyomial of degree 1 ca have may poit-value represetatios. A poit-value represetatio, however, describes exactly oe polyomial. Let A(x) = 3x 3 + 2x 2 + x + 1. Some of its poit-value represetatios are the vectors ad {(0, 1), (1, 7), (2, 35), (3, 103)} {(0, 1), (1, 7), ( 1, 1)(2, 35)} Those vectors however describe oly the polyomial A(x). Give the poit-value represetatios of polyomials {(x 0, y 0 ), (x 1, y 1 ),..., (x 1, y 1 )} {(x 0, y 0), (x 1, y 1),..., (x 1, y 1)} of degree 1 where y i = A(x i ) ad y i = B(x i) for i = 0, 1,..., 1, additio ca be doe i Θ() time by producig the poit-value represetatio: {(x 0, y 0 + y 0), (x 1, y 1 + y 1),..., (x 1, y 1 + y 1)} MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 13/42 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 15/42 Computig a poit-value represetatiopoit-value represetatio - Multiplicatio Give a polyomial A(x) of degree 1 havig the form: we compute its poit-value represetatio usig Horer s method: 1. Select distict poits x 0, x 1,..., x 1. 2. Evaluate A(x i ) for i = 0, 1,..., 1. 3. Poit-value represetatio is the set 1 (x i, A(x i )) Evaluatig a polyomial at poits takes Θ( 2 ) time. Give the poit-value represetatios of polyomials {(x 0, y 0 ), (x 1, y 1 ),..., (x 1, y 1 )} {(x 0, y 0), (x 1, y 1),..., (x 1, y 1)} of degree 1, where y i = A(x i ) ad y i = B(x i) for i = 0, 1,..., 1, we must exted the poit-value represetatios of A(x) ad B(x) to 2 1 pairs as the polyomial resultig from multiplicatio is of degree 2 2.
Poit-value represetatio - MultiplicatioPoit-value represetatio - Iterpolatio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 18/42 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 20/42 Give the poit-value represetatios of polyomials {(x 0, y 0 ), (x 1, y 1 ),..., (x 2 2, y 2 2 )} {(x 0, y 0), (x 1, y 1),..., (x 2 2, y 2 2)} of degree 1 where y i = A(x i ) ad y i = B(x i) for i = 0, 1,..., 2 2, multiplicatio ca be doe i Θ() time by producig the poit-value represetatio: LAGRANGE S FORMULA Give the poit-value represetatios {(x 0, y 0 ), (x 1, y 1 ),..., (x 1, y 1 )} of a polyomial A(x), we ca compute its coefficiet represetatio i time Θ( 2 ) usig the formula: 1 (x x j ) j i A(x) = y i (x i x j ) j i {(x 0, y 0 y 0), (x 1, y 1 y 1),..., (x 2 2, y 1 y 2 2)} MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 17/42 Poit-value represetatio - Iterpolatio Iterpolatio is the process of obtaiig the coefficiet represetatio of a polyomial, give its poit-value represetatio. Some methods to do this are: 1. Solve a set of equatios usig LU decompositio. Takes O( 3 ) time. 2. Use Lagrage s formula. Takes O( 2 ) time. MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 19/42 Poit-value represetatio - Summary Evaluatio (Costructio) i Θ( 2 ) Additio i Θ() Multiplicatio i Θ() Iterpolatio i Θ( 2 )
Compariso of the two represetatios MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 22/42 Solutio Speed up trasformatio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 24/42 COEFFICIENT REPRESENTATION Evaluatio i Θ() Additio i Θ() Multiplicatio i Θ( 2 ) POINT-VALUE REPRESENTATION Evaluatio (Costructio) i Θ( 2 ) Additio i Θ() Multiplicatio i Θ() Iterpolatio i Θ( 2 ) a 0, a 1,..., a 1 b 0, b 1,..., b 1 Evaluatio Time Θ( log ) (x 0, y 0 ),..., (x 2 2, y 2 2 ) (x 0, y 0 ),..., (x 2 2, y 2 2 ) Ordiary multiplicatio Time Θ( 2 ) Poitwise multipl. Time Θ() Iterpolatio Time Θ( log ) c 0, c 1,..., c 2 2 (x 0, y 0 y 0 ) (x 1, y 1 y 1 ). (x 2 2, y 2 2 y 2 2 ) MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 21/42 Problem Multiplicatio is too slow MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 23/42 Complex roots of uity A complex th root of uity is a complex umber ω such that a 0, a 1,..., a 1 b 0, b 1,..., b 1 Evaluatio Time Θ( 2 ) (x 0, y 0 ),..., (x 2 2, y 2 2 ) (x 0, y 0 ),..., (x 2 2, y 2 2 ) Ordiary multiplicatio Time Θ( 2 ) Poitwise multipl. Time Θ() c 0, c 1,..., c 2 2 Time Θ( 2 ) Iterpolatio (x 0, y 0 y 0 ) (x 1, y 1 y 1 ). (x 2 2, y 2 2 y 2 2 ) ω = 1 There are exactly complex roots of uity: ω k = e 2πik k = 0, 1,..., 1. It holds that e iu = cos(u) + i si(u). ω 3 8 ω 2 8 ω 1 8 ω 4 8 ω 8 8 for ω 5 8 ω 6 8 ω 7 8
MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 26/42 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 28/42 Properties of complex roots of uity The complex th roots of uity form a group uder multiplicatio that has the same structure as the additive group (Z, +) modulo, sice ω = ω 0 = 1 imples that ω j ω k = ω j+k = ω (j+k) mod Corollary For ay eve iteger > 0, Proof ω 2 = e 2πi 2 = e πi = cos(π) + i si(π) = 1. Similarly, ω 1 ω 2 = ω 2 = 1 = ω 1. MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 25/42 Properties of complex roots of uity Lemma f > 0 is eve, the the squares of the complex th roots of uity are the /2 complex (/2)th roots of uity. Proof From Corollary 1: ω 2 = 1 ω 2 ω k = ω k (ω k+ 2 ) 2 = (ω k ) 2 This is the most importat lemma cocerig FFT. Discrete Fourier Trasformatio The Discrete Fourier Trasformatio of a coefficiet vector of polyomial is the vector where Notio: y = DFT (a) α = (a 0, a 1,..., a 1 ) y = (y 0, y 1,..., y 1 ) y k = A(ω k ), k = 0, 1,..., 1 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 27/42 Fast Fourier Trasformatio The Fast Fourier Trasform is a method for computig the Discrete Fourier Trasformatio by employig a divide-ad-coquer strategy which: Takes advatage of the properties of complex roots of uity. Takes time Θ( log ) istead of Θ( 2 ). Requires that is a exact power of 2 (at least the basic algorithm). We ca covert ay polyomial of degree 1 1 where k so that 2 k = 1 to a polyomial of degree 2 1 where 2 is a exact power of 2 ( k so that 2 k = 2 ) by appedig coefficiets a 1,..., a 2 1 equal to 0.
Computig DFT usig FFT MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 30/42 Computig DFT usig FFT MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 32/42 Suppose we have a polyomial where is a exact power of 2. We ca divide our origial polyomial i two polyomial A [0] (x) ad A [1] (x) of degree /2 1. A [0] (x) = a 0 + a 2 x + a 4 x 2 +... + a 2 x /2 1 (a 0, a 1, a 2, a 3, a 4, a 5, a 6, a 7 ) (a 0, a 2, a 4, a 6 ) (a 1, a 3, a 5, a 7 ) (a 0, a 4 ) (a 2, a 6 ) (a 1, a 5 ) (a 3, a 7 ) (a 0 ) (a 4 ) (a 2 ) (a 6 ) (a 1 ) (a 5 ) (a 3 ) (a 7 ) where A [1] (x) = a 1 + a 3 x + a 5 x 2 +... + a 1 x /2 1 A(x) = A [0] (x 2 ) + xa [1] (x 2 ) T () = 2T (/2) + Θ() = Θ( lg ) where MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 29/42 Computig DFT usig FFT A [0] (x) = a 0 + a 2 x + a 4 x 2 +... + a 2 x /2 1 A [1] (x) = a 1 + a 3 x + a 5 x 2 +... + a 1 x /2 1 A(x) = A [0] (x 2 ) + xa [1] (x 2 ) (1) So the problem of evaluatig A(x) at ω, 0 ω, 1..., ω 1 is reduced to: 1. Evaluatig the polyomials A [0] (x) ad A [1] (x) of degree /2 1 at the poits (ω 0 ) 2, (ω 1 ) 2,..., (ω 1 ) 2 (2) 2. Combig the results accordig to (1). By the Halvig lemma, the list of values (2) cosists ot of distict values but oly of the /2 complex (/2)th roots of uity. y 0 y 1 y 2 y 3. y 1 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 31/42 Iterpolatio 1 1 1 1 1 1 ω ω 2 ω 3 ω 1 1 ω = 2 ω 4 ω 6 ω 2( 1) 1 ω 3 ω 6 ω 9 ω 3( 1)........ 1 ω 1 ω 2( 1) ω 3( 1) ω ( 1)( 1) a 0 a 1 a 2 a 3. a 1 For evaluatio we have y = DFT (a). For iterpolatio we have the Iverse Discrete Fourier Trasformatio a = DFT 1 (Y ) For the iverse operatio we proceed by multiplyig y by the matrix V 1, the iverse of V (Vadermode matrix). Theorem For j, k = 0, 1,..., 1, the (j, k) etry of V 1 is ω kj /.
Iterpolatio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 34/42 The strig matchig with mismatches MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 36/42 Give the iverse matrix V 1 give by the formula a i = 1 for j = 0, 1,..., 1. Recall the formula for DFT (a), we have that DFT 1 (y) is k=0 y i = k=0 y k ω ki a k ω ki This meas that, by modifyig the FFT algorithm to switch the roles of a ad y, replace ω by ω 1, ad divide each elemet of the result by, we ca compute the iverse DFT i Θ( lg ) time as well. Defiitio Give a patter P = p 1 p 2... p m ad a text T = t 1 t 2... t, fid the umber of matched characters at each positio i i the text, where 0 < i m. Let P = GT C ad T = GGACT. Text: G G A C T Matches: 1 2 0 The time complexity of the aive algorithm is O(m). MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 33/42 Iterpolatio MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 35/42 The strig matchig with mismatches Theorem (Covolutio theorem) For ay two vectors a ad b of legth, where is a power of 2, it holds that a b = DFT 1 2 (DFT 2(a) DFT 2 (b)) where the vectors a ad b are padded with 0s to legth 2 ad deotes the compoetwise product of two 2-elemet vectors. ALGORITHM Give a patter P = p 1 p 2... p m ad a text T = t 1 t 2... t over a alphabet Σ 1. For each x Σ (a) Compute the vectors T x of legth, where T x [i] is 1 if T [i] = x otherwise 0, where 0 < i. (b) Compute the vectors P x of legth m, where P x [i] is 1 if P [i] = x otherwise 0, where 0 < i m. (c) Compute the vectors C x as the covolutio of T x ad the reverse of P x. 2. Compute the vector C = C x for all x Σ.
The strig matchig with mismatches MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 38/42 The strig matchig with mismatches MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 40/42 T = GGACT P = GT C T G = (1, 1, 0, 0, 0) P G = (1, 0, 0) 1 1 0 0 0 0 0 1 1 1 0 0 0 0 0 1 1 0 0 0 T = GGACT P = GT C T C = (0, 0, 0, 1, 0) P C = (0, 0, 1) 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 The umbers of the product deote the umber of matched Gs at each positio i the text. G G A C T The umbers of the product deote the umber of matched Cs at each positio i the text. G G A C T MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 37/42 The strig matchig with mismatches MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 39/42 The strig matchig with mismatches 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 0 T = GGACT P = GT C T A = (0, 0, 1, 0, 0) P A = (0, 0, 0) T = GGACT P = GT C T T = (0, 0, 0, 0, 1) P T = (0, 1, 0) 0 0 0 0 1 0 0 1 0 The umbers of the product deote the umber of matched As at each positio i the text. G G A C T The umbers of the product deote the umber of matched T s at each positio i the text. G G A C T
The strig matchig with mismatches The strig matchig with mismatches 0 0 1 1 0 0 0 0 0 + 0 0 0 1 0 0 0 1 0 0 0 1 2 0 1 0 G G A C T The time complexity of this algorithm is O( Σ lg ). We ca use a more efficiet implemetatio of FFT, where the two polyomials do ot have to be of degree 1 where is a exact power of 2 ad the time complexity improves to O( Σ lg m). The umbers of the sum deote the umber of matched characters at each positio i the text. MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 41/42 MI-EVY (2011/2012) J. Holub: 4. DFT, FFT ad Patter Matchig p. 42/42