2. ALGORITHM ANALYSIS

Transcription

1 2. ALGORITHM ANALYSIS computatioal tractability survey of commo ruig times 2. ALGORITHM ANALYSIS computatioal tractability survey of commo ruig times Lecture slides by Kevi Waye Copyright 2005 Pearso-Addiso Wesley Copyright 2013 Kevi Waye SECTION 2.1 Last updated o Sep 8, :19 AM A strikigly moder thought Brute force As soo as a Aalytic Egie exists, it will ecessarily guide the future course of the sciece. Wheever ay result is sought by its aid, the questio will arise By what course of calculatio ca these results be arrived at by the machie i the shortest time? Charles Babbage (1864) Brute force. For may otrivial problems, there is a atural brute-force search algorithm that checks every possible solutio. Typically takes 2 time or worse for iputs of size. Uacceptable i practice. how may times do you have to tur the crak? Aalytic Egie 3 4

2 Polyomial ruig time Polyomial ruig time Desirable scalig property. Whe the iput size doubles, the algorithm should oly slow dow by some costat factor C. Def. A algorithm is poly-time if the above scalig property holds. We say that a algorithm is efficiet if has a polyomial ruig time. Justificatio. It really works i practice! I practice, the poly-time algorithms that people develop have low costats ad low expoets. Breakig through the expoetial barrier of brute force typically exposes some crucial structure of the problem. There exists costats c > 0 ad d > 0 such that o every iput of size, its ruig time is bouded by c d primitive computatioal steps. choose C = 2 d Exceptios. Some poly-time algorithms do have high costats ad/or expoets, ad/or are useless i practice. Map graphs i polyomial time Q. Which would you prefer vs l? Mikkel Thorup Departmet of Computer Sciece, Uiversity of Copehage Uiversitetsparke 1, DK-2100 Copehage East, Demark mthorup@diku.dk Abstract vo Neuma (1953) Nash (1955) Gödel (1956) Cobham (1964) Edmods (1965) Rabi (1966) Che, Grigi, ad Papadimitriou (WADS 97 ad STOC 98) have itroduced a modified otio of plaarity, where two faces are cosidered adjacet if they share at least oe poit. The correspodig abstract graphs are called map graphs. Che et.al. raised the questio of whether map graphs ca be recogized i polyomial time. They showed that the decisio problem is i NP ad preseted a polyomial time algorithm for the special case where we allow at most 4 faces to itersect i ay poit if oly 3 are allowed to itersect i a poit, we get the usual plaar graphs. Che et.al. cojectured that map graphs ca be recogized i polyomial time, ad i this paper, their cojecture is settled affirmatively. 5 6 Worst-case aalysis Types of aalyses Worst case. Ruig time guaratee for ay iput of size. Geerally captures efficiecy i practice. Dracoia view, but hard to fid effective alterative. Exceptios. Some expoetial-time algorithms are used widely i practice because the worst-case istaces seem to be rare. Worst case. Ruig time guaratee for ay iput of size. Ex. Heapsort requires at most 2 log2 compares to sort elemets. Probabilistic. Expected ruig time of a radomized algorithm. Ex. The expected umber of compares to quicksort elemets is ~ 2 l. Amortized. Worst-case ruig time for ay sequece of operatios. Ex. Startig from a empty stack, ay sequece of push ad pop operatios takes O() operatios usig a resizig array. Average-case. Expected ruig time for a radom iput of size. Ex. The expected umber of character compares performed by 3-way radix quicksort o uiformly radom strigs is ~ 2 l. simplex algorithm Liux grep k-meas algorithm Also. Smoothed aalysis, competitive aalysis,

3 Why it matters 2. ALGORITHM ANALYSIS computatioal tractability survey of commo ruig times SECTION Big-Oh otatio Notatioal abuses Upper bouds. T() is O( f ()) if there exist costats c > 0 ad 0 0 such that T() c f () for all 0. Ex. T() = T() is O( 2 ). T() is also O( 3 ). choose c = 50, 0 = 1 T() is either O() or O( log ). Typical usage. Isertio makes O( 2 ) compares to sort elemets. 0 c f () T() Equals sig. O( f ()) is a set of fuctios, but computer scietists ofte write T() = O( f ()) istead of T() O( f ()). Ex. Cosider f () = 5 3 ad g () = 3 2. We have f () = O( 3 ) = g(). Thus, f () = g(). Domai. The domai of f () is typically the atural umbers { 0, 1, 2,. Sometimes we restrict to a subset of the atural umbers. Other times we exted to the reals. Alterate defiitio. T() is O( f ()) if sup T () f() <. Noegative fuctios. Whe usig big-oh otatio, we assume that the fuctios ivolved are (asymptotically) oegative. Bottom lie. OK to abuse otatio; ot OK to misuse it

4 Big-Omega otatio Big-Theta otatio Lower bouds. T() is Ω( f ()) if there exist costats c > 0 ad 0 0 such that T() c f () for all 0. Ex. T() = T() is both Ω( 2 ) ad Ω(). T() is either Ω( 3 ) or Ω( 3 log ). choose c = 32, 0 = 1 T() c f () Tight bouds. T() is Θ( f ()) if there exist costats c1 > 0, c2 > 0, ad 0 0 such that c1 f () T() c2 f () for all 0. Ex. T() = T() is Θ( 2 ). T() is either Θ() or Θ( 3 ). choose c1 = 32, c2 = 50, 0 = 1 c2 f () T() c1 f () 0 0 Typical usage. Ay compare-based sortig algorithm requires Ω( log ) compares i the worst case. Typical usage. Mergesort makes Θ( log ) compares to sort elemets. Meaigless statemet. Ay compare-based sortig algorithm requires at least O( log ) compares i the worst case Useful facts Propositio. If f(), the f () is Θ(g()). g() = c > 0 Pf. By defiitio of the it, there exists 0 such such that for all c < f() g() < 2 c Thus, f () 2 c g() for all 0, which implies f () is O(g()). Similarly, f () ½ c g() for all 0, which implies f () is Ω(g()). Asymptotic bouds for some commo fuctios Polyomials. Let T() = a 0 + a a d d with a d > 0. The, T() is Θ( d ). Pf. a 0 + a a d d d = a d > 0 Logarithms. Θ(log a ) is Θ(log b ) for ay costats a, b > 0. Logarithms ad polyomials. For every d > 0, log is O( d ). o eed to specify base (assumig it is a costat) Propositio. If f() g() = 0, the f () is O(g()). Expoetials ad polyomials. For every r > 1 ad every d > 0, d is O(r ). Pf. d r =

5 Big-Oh otatio with multiple variables Upper bouds. T(m, ) is O( f (m, )) if there exist costats c > 0, m 0 0, ad 0 0 such that T(m, ) c f (m, ) for all 0 ad m m ALGORITHM ANALYSIS Ex. T(m, ) = 32m m T(m, ) is both O(m ) ad O(m 3 ). T(m, ) is either O( 3 ) or O(m 2 ). computatioal tractability survey of commo ruig times Typical usage. Breadth-first search takes O(m + ) time to fid the shortest path from s to t i a digraph. SECTION Liear time: O() Liear time: O() Liear time. Ruig time is proportioal to iput size. Computig the maximum. Compute maximum of umbers a 1,, a. Merge. Combie two sorted lists A = a 1, a 2,, a with B = b 1, b 2,, b ito sorted whole. max a 1 for i = 2 to { if (a i > max) max a i i = 1, j = 1 while (both lists are oempty) { if (a i b j ) apped a i to output list ad icremet i else(a i b j )apped b j to output list ad icremet j apped remaider of oempty list to output list Claim. Mergig two lists of size takes O() time. Pf. After each compare, the legth of output list icreases by

6 Liearithmic time: O( log ) Quadratic time: O( 2 ) O( log ) time. Arises i divide-ad-coquer algorithms. Ex. Eumerate all pairs of elemets. Sortig. Mergesort ad heapsort are sortig algorithms that perform O( log ) compares. Closest pair of poits. Give a list of poits i the plae (x 1, y 1 ),, (x, y ), fid the pair that is closest. Largest empty iterval. Give time-stamps x 1,, x o which copies of a file arrive at a server, what is largest iterval whe o copies of file arrive? O( log ) solutio. Sort the time-stamps. Sca the sorted list i order, idetifyig the maximum gap betwee successive time-stamps. O( 2 ) solutio. Try all pairs of poits. mi (x 1 - x 2 ) 2 + (y 1 - y 2 ) 2 for i = 1 to { for j = i+1 to { d (x i - x j ) 2 + (y i - y j ) 2 if (d < mi) mi d Remark. Ω( 2 ) seems ievitable, but this is just a illusio. [see Chapter 5] Cubic time: O( 3 ) Polyomial time: O( k ) Cubic time. Eumerate all triples of elemets. Set disjoitess. Give sets S 1,, S each of which is a subset of 1, 2,,, is there some pair of these which are disjoit? Idepedet set of size k. Give a graph, are there k odes such that o two are joied by a edge? O( k ) solutio. Eumerate all subsets of k odes. k is a costat O( 3 ) solutio. For each pair of sets, determie if they are disjoit. foreach set S i { foreach other set S j { foreach elemet p of S i { foreach subset S of k odes { check whether S i a idepedet set if (S is a idepedet set) report S is a idepedet set determie whether p also belogs to S j if (o elemet of S i belogs to S j ) report that S i ad S j are disjoit Check whether S is a idepedet set takes O(k 2 ) time. Number of k elemet subsets = ( 1)( 2) ( k + 1) = O(k 2 k / k!) = O( k k k(k 1)(k 2) 1 ). poly-time for k=17, but ot practical k k! 23 24

7 Expoetial time Subliear time Idepedet set. Give a graph, what is maximum cardiality of a idepedet set? Search i a sorted array. Give a sorted array A of umbers, is a give umber x i the array? O( 2 2 ) solutio. Eumerate all subsets. O(log ) solutio. Biary search. S* φ foreach subset S of odes { check whether S i a idepedet set if (S is largest idepedet set see so far) update S* S lo 1, hi while (lo hi) { mid (lo + hi) / 2 if (x < A[mid]) hi mid - 1 else if (x > A[mid]) lo mid + 1 else retur yes retur o 25 26