CS270 Combinatorial Algorithms & Data Structures Spring Lecture 9:

Transcription

1 CS70 Comiatorial Algorithms & Data Structures Sprig 00 Lecture 9: 170 Lecturer: Satish Rao Scrie: Adam Chlipala Disclaimer: These otes have ot ee sujected to the usual scrutiy reserved for formal pulicatios They may e distriuted outside this class oly with the permissio of the Istructor 91 Liear programmig wrap-up 911 Seidel s algorithm Give the liear program Ax ; maxc with d variales ad costraits: Algorithm 1 Seidel(A,, c) 1: if there is oly oe variale the : Treat all iequalities i Ax as equalities ad solve for x i each 3: Retur the poit out of these that maximizes c : ed if 5: if there are as may costraits as variales the 6: Solve the liear system Ax = usig Gaussia elimiatio ad retur the resultig poit 7: ed if 8: Pick a costrait h : a i x i uiformly at radom 9: x Seidel(A a i, i, c) 10: if x satisfies h the 11: Retur x 1: else 13: A, sustitutio of the equality a i x = i i Ax 1: Retur Seidel(A,, c) 15: ed if 9111 Correctess Lies 1 through 7 solve the particularly simple cases that have oly oe variale or as may costraits as variales The remaiig code recurses util oe of these special cases is reached Clearly lie 11 will oly retur correct values: We iductively assume that x is the est solutio without the chose h Reicludig h could oly restrict the solutio space, ut x is verified to satisfy h, so it must e the solutio to the origial prolem Figure 91 illustrates this case If lie 1 is reached, the x did ot satisfy h This meas h is oe of the importat tight costraits that determies the solutio Therefore, we kow that the solutio lies o h s hyperplae, so a i x = i for the true solutio x This equatio may e simplified to oe with oly oe of the variales o the lefthad side, allowig the sustitutio of the righthad side for that variale i A ad, lowerig the dimesioality of the prolem We kow that the ew system is equivalet to the old, so, iductively, the retur value of the recursive ivocatio is the correct solutio Figure 9 illustrates this case 9-1

2 c x h Figure 91: Removig a otight costrait c x h Figure 9: Removig a tight costrait

3 911 Ruig time Deote y T(, d) the expected ruig time of Seidel s algorithm o a iput with d variales ad costraits Aside from the ase cases, Seidel will always make oe recursive call with a system of oe fewer costrait If the radomly chose h is a tight costrait, it will also make a recursive call with oe fewer variale There are exactly d tight costraits, so there is a d chace of this recursive call occurrig Thus, cosiderig that the time for recursio will domiate the time spet y the ase cases, we arrive at this recurrece: T(, d) = T( 1, d) + d T(, d 1) There are simple methods that ca e used for solvig commo recurreces, ut they do t apply to this oe! It turs out that a solutio for this oe is: T(, d) = O(d!) Thus, if we hold d fixed, Seidel s algorithm rus i expected liear time 91 Summary of liear programmig We ve looked at these algorithms: Simplex While it performs very well i practice, it does t ru i polyomial time However, it has recetly ee prove that a slight perturatio of ay family of liear programs leads to a family for which Simplex rus i polyomial time Karmarkar s iterior poit method Both polyomial time ad efficiet i practice The ellipsoid method The first polyomial time algorithm discovered, ut very iefficiet i practice However, it geeralizes to oliear systems ad is useful i reasoig aout may kids of prolems Seidel s algorithm Radomized algorithm that is efficiet for prolems with low fixed dimesioalities Liear programmig is very useful for game theoretic prolems May, may other iterestig prolems ca e reduced to liear programmig prolems 9 Recurreces 91 A example recurrece Cosider this recurrece: T() = T(1) = 1 T ( ) + We ve all see this oe efore for algorithms like merge sort, so the solutio T() = O( log ) comes to mid We ca verify this y guessig T() = a log for some a ad verifyig that the solutio has the

4 right properties: T ( ( ( ) + = a log + ) ) = a(log log ) + = a log a + = a log (a 1) a log = T() However, there s a simple procedure for solvig may divide-ad-coquer recurreces directly Cosider the recursio tree for a algorithm with ruig time descried y the T aove, with every ode laeled with the amout of work doe i it as opposed to the recursive calls it makes: 1() ( ) ( ) d ( d ) The umers i the righthad colum give the total work for each level of the tree, with the last row givig the geeral term for the sum of level d It is clear that the total work o each level is The umer of levels is O(log ) Thus, summig up the total amout of work, we arrive at a O( log ) oud 9 Geeral divide-ad-coquer recurreces This sort of reasoig geeralizes to recurreces of the form T() = at ( ) + for ay a, : a a 1() a ( ) a a ( ) d d For the geeral form, we have a rachig factor of a with d doe at every ode at level d Thus, we have: T() = log If a <, the a < 1, ad the sum is a prefix of a coverget geometric series, makig the sum O(1) for fixed a ad This meas T() = O() i=0 ( a ) i

5 If a =, the we have the same sort of recursio as i the origial example, yieldig T() = O( log ) Fially, if a >, the a > 1, ad the last term of the sum domiates asymptotically, so T() = O ( ( a ) log ) The time expressio simplifes to a log, ad usig the trick of switchig the ase ad log parameter, we have T() = O( log a ) 93 The Master s Theorem Now cosider the recurrece T() = at ( ) + By usig the sustitutio m =, we ca use the result from the last sectio to solve this recurrece Let S(m) = T() We otai: ( m ) S(m) = as + m Now the last sectio s result ca e used directly to fid S From there, we otai T() = S( ) Geeralizig this approach to aritrary expoets for, we ca otai the well-kow Master s Theorem This approach also works for recurreces that make recursive calls with differet size parameters For example, for ( ( ) T() = T + T + 3) 3 the recursio tree ca e see to look a lot like that for the first recurrece we cosidered The rachig factor is, ad the depth is log /3, leadig to aother T() = O( log ) coclusio 93 Clever multiplicatio Stadard grade-school multiplicatio of two -it itegers takes O( ) time, ut it is possile to do etter Cosider this decompositio of two umers x ad y to e multiplied: x y a c d We ca treat the multiplicatio of x ad y as multiplicatio of two two-digit ase- / umers I other words: xy = ac + c / + ad / + d Now these su-multiplicatios may e decomposed i the same way, dow to the poit where we oly wish to multiply sigle-digit umers This leads to the recurrece ( T() = T + ) Ufortuately, y the Master s Theorem, this oly gives T() = O( ), which is o etter tha what the aive algorithm achieves However, we ca use a clever trick iveted y five-year-old Gauss to do etter

6 We eed oly perform three multiplicatios: (a + )(c + d), d, ad ac The last two give us two of the four products we eed to fid the fial result, with ac shifted y i costat time to otai ac By sutractig d ad ac from the first product, we otai c + ad Shiftig this y i costat time, we otai c / + ad / Now we ca add all the terms we ve otaied so far to otai the aswer With this strategy, oly 3 multiplicatios are performed per recursio tree ode, chagig the recurrece to ( T() = 3T + ) So ow T() = O( log 3 ), so T() = O( 16 ), which makes a sigificat differece i practice It s possile to do eve etter tha this with similar techiques Karatsua s algorithm for multiplyig matrices is ased o this idea 9 Computig medias Defiitio 91 The media of a sequece S is the S th lowest elemet of the sequece Oe ovious way to fid a media of a sequece is to first sort it i O( log ) time ad the read the middle elemet of the sequece i O(1) time However, it is possile to do etter We ll use a divide-ad-coquer approach that solves a more geeral prolem: Defiitio 9 The selectio prolem is to, give a sequece S ad idex k etwee 0 ad S 1, fid the kth lowest elemet of S Algorithm Select(A, k) if k = 0 the Retur mi(a) else Choose x A uiformly at radom B y A : y x C y A : y > x if k < B the Retur Select(B, k) else Retur Select(C, k B ) ed if ed if The correctess of the algorithm is est argued with a diagram: A B k x C k B A sequece A is show divided ito the B ad C that Select would form for a chose x, with B ad C cosidered to e sorted If the k < B, the A[k] must e i B, sice B cotais at least the k + 1 lowest

7 elemets of A Furthermore, B must e the kth lowest elemet of B as well, so the first recursive call will always yield the correct retur Similarly, if k B, the A[k] must ed up i C, ad it must e the (k B )th lowest, sice the B lowest elemets of A have ee moved to B Thus, Select is correct Lettig T e the ruig time of Select o a A of legth We have: E[ T()] E[ T(γ)] + where γ is the proportio of A s elemets that ed up i the susequece that s used i the recursive call We might thik we ca proceed with E[ T()] E[ T(γ)] + E[ T(E[γ])] + β Pr[γ = β]e[ T(β)] + ut the secod step is ivalid Ituitively, we see that E[γ] 3 If we could sustitute this i the recurrece, we would get E[ T()] E [ T ( 3 )] + ad fid that T() = O() is a solutio However, we ca actually otai this result formally through a slight modificatio to the algorithm Algorithm 3 Select(A, k) if k = 0 the Retur mi(a) else repeat Choose x A uiformly at radom B y A : y x C y A : y > x util mi{ B, C } > A if k < B the Retur Select(B, k) else Retur Select(C, k B ) ed if ed if With a atagoistic radom umer geerator, the ew algorithm may ever termiate, ut most likely it will termiate, ad we kow that γ will ever e higher tha 3 Therefore: [ ( )] 3 E[ T()] E T + E[ d] where d is the umer of iteratios of the loop required efore a suitale x is foud Half of the x values are suitale, so E[ d] =, ad we otai E[ T()] = O() y the Master s Theorem The ruig time of Select is actually a upper oud o the ruig time of Select All of the work for these algorithms is doe i partitioig iput sequeces Whe Select chooses a x such that mi{ B, C } > A A, the it ehaves just like Select for that step If mi{ B, C } ad k is i the smaller susequece, the Select ca oly do etter tha Select does, sice Select makes its recursive call with a larger sequece I the fial case, mi{ B, C } A ad k is i the larger susequece Here, Select s recursive call uses a larger sequece tha Select s, ut Select s first repetitio of its loop whe Select s choice of x is picked

8 Figure 93: Liear system B a 0 a 1 a Figure 9: Liear system A first does the same work as Select s recursive call does I effect, the work doe y Select is pushed up ito the loops of Select util the property that recursive calls are oly made o arrays o larger tha 3 of the iput sequece size is met After such a trasformatio, every executio of Select ecomes a executio of Select, ad so the O() expected time oud applies 95 The Fast Fourier Trasform 951 Motivatio Cosider a cotiuous liear system B like that i Figure 93 The x axis represets time ad the y axis represets somethig like the eergy i the system at that time The cotiuous system is approximated with a discrete system through samplig, represeted y arrows i the figure Whe a istat force is applied to B, the shape of the resultig system is B s shape, ut the magitude is proportioal to the magitude of the force Whe a cotiuous force like A i Figure 9 is applied, the result is the superpositio of the differet results of istat forces withi A Samplig A discretely as well, we arrive at the decompositio show i Figure 95 for the system resultig from applyig A to B The

9 Figure 95: Decompositio of the result of applyig A to B eergy i the system at each time is the sum of the eergies of the differet parts of the decompositio active at that time Each decompositio correspods with a istat force withi A, ad each starts oe time uit later tha the last Now cosider geeral systems A ad B of the types cosidered aove, sampled over t time uits Deote y C the system resultig from the applicatio of A to B There is a simple patter i the equatios for the itesities of C at differet time uit: c 0 = a 0 0 c 1 = a a 1 0 c = a 0 + a a 0 The aive algorithm for calculatig C takes O(t ) multiplicatios, oe for each pair of itesities etwee A ad B It turs out that we ca do etter tha this Cosider the system A as a polyomial A(x) = a 0 + a 1 x + a x + + a t 1 x t 1 Defie B(x) aalogously, ad let C(x) = A(x) B(x) Upo ispectig the polyomial C, it ecomes clear that its coefficiets are exactly the c i values that we re lookig for Now that we ve reduced the prolem to multiplicatio of polyomials, we ca use the Fast Fourier Trasform algorithm to fid C i O(t log t) 95 Outlie of the algorithm There are two mai strategies used for represetig polyomials The oe we ve used so far represets a degree polyomial y its + 1 coefficiets Alteratively, a degree polyomial is also defied y + 1 distict poits

10 This meas we ca represet A(x) y (x 0, A(x 0 )),, (x t 1, A(x t 1 ) for distict x 0,, x t 1 If we represet B(x) i the same maer with the same x i s, the C(x) is (x 0, A(x 0 ) B(x 0 )),, (x t 1, A(x t 1 ) B(x t 1 )) Whe we have the ew forms of A ad B, the ew form of C ca e foud with O(t) multiplicatios Motivated y this possiility, we d like to covert etwee polyomial represetatios as ecessary to take advatage of it We ca propose the followig shell for a algorithm: Algorithm FFT(A, B) 1: Choose distict x 0,, x t 1 : Evaluatio: Calculate A(x i ) ad B(x i ) for each 0 i < t 3: Calculate C(x i ) for each 0 i < t y multiplyig A(x i ) B(x i ) : Iterpolatio: Determie C from its values at the x i s Lies 1 ad 3 take O(t) time To eat O(t ) overall ruig time, we ll eed to fid ways to do lies ad i o(t ) time Oe way to thik aout evaluatio is as computig the result of the followig matrix multiplicatio: 1 x 0 x 0 x t x t 1 x t 1 x t 1 t 1 a 0 a t 1 = A(x 0 ) A(x t 1 ) Iterpolatio ca e viewed very similarly Left multiplyig oth sides of this equatio y the iverse of the matrix yields a equatio for fidig C from eough poits through aother matrix multiplicatio The aive way of performig these multiplicatios takes O(t ) time However, if we ca fid a etter way that takes advatage of the special structure of the matrix, the we ca eat the aive polyomial multiplicatio algorithm I fact, the FFT does just this To e cotiued