CS 270 Algorithms. Oliver Kullmann. Growth of Functions. Divide-and- Conquer Min-Max- Problem. Tutorial. Reading from CLRS for week 2

Transcription

1 Geeral remarks Week 2 1 Divide ad First we cosider a importat tool for the aalysis of algorithms: Big-Oh. The we itroduce a importat algorithmic paradigm:. We coclude by presetig ad aalysig two examples. 2 Readig from CLRS for week 2 Chapter 2.3 Chapter 3 3 O-Notatio A way to describe behaviour of fuctios i the limit. We are studyig asymptotic efficiecy. Describe growth of fuctios. Focus o what s importat by abstractig away low-order terms ad costat factors. How we idicate ruig times of algorithms. A way to compare sizes of fuctios: O correspods to Ω correspods to Θ correspods to = O ( g() ) is the set of all fuctios f () for which there are positive costats c ad 0 such that f () cg() for all 0. 0 cg() f () We cosider oly fuctios f, g : N R 0. g() is a asymptotic upper boud for f (). If f () O(g()), we write f () = O(g()) (we will precisely explai this soo)

2 O-Notatio Examples Ω-Notatio 2 2 = O( 3 ), with c = 1 ad 0 = 2. Example of fuctios i O( 2 ): Ω ( g() ) is the set of all fuctios f () for which there are positive costats c ad 0 such that f () cg() for all Also f () cg() / / lg lg lg g() is a asymptotic lower boud for f (). Ω-Notatio Examples Θ-Notatio = Ω(lg ), with c = 1 ad 0 = 16. Example of fuctios i Ω( 2 ): Θ ( g() ) is the set of all fuctios f () for which there are positive costats c 1, c 2 ad 0 such that c 1 g() f () c 2 g() for all c 2 g() Also f () c 1 g() lg lg lg g() is a asymptotic tight boud for f (). 2 2

3 Θ-Notatio (cot d) Asymptotic otatio i equatios Whe o right-had side Examples 1 2 /2 2 = Θ( 2 ), with c 1 = 1 4, c 2 = 1 2, ad 0 = 8. Θ( 2 ) stads for some aoymous fuctio i the set Θ( 2 ) = Θ() meas = f () for some f () Θ(). I particular, f () = Theorem 2 f () = Θ(g()) if ad oly if f () = O(g()) ad f () = Ω(g()). Leadig costats ad lower order terms do ot matter. Whe o left-had side No matter how the aoymous fuctios are chose o the left-had side, there is a way to choose the aoymous fuctios o the right-had side to make the equatio valid. Iterpret Θ() = Θ( 2 ) as meaig for all fuctios f () Θ(), there exists a fuctio g() Θ( 2 ) such that f () = g(). Asymptotic otatio chaied together Example Aalysis = Θ() = Θ( 2 ) Iterpretatio: First equatio: There exists f () Θ() such that = f (). Secod equatio: For all g() Θ() (such as the f () used to make the first equatio hold), there exists h() Θ( 2 ) such that g() = h(). Isertio-Sort(A) 1 for j = 2 to A.legth 2 key = A[j] 3 / Isert A[j] ito sorted sequece A[1.. j 1]. 4 i = j 1 5 while i > 0 ad A[i] > key 6 A[i+1] = A[i] 7 i = i 1 8 A[i+1] = key Note What has bee said of Θ o this ad the previous slide also applies to O ad Ω. The for -loop o lie 1 is executed O() times; ad each statemet costs costat time, except for the while -loop o lies 5-7 which costs O(). Thus overall rutime is: O() O() = O( 2 ). Note: I fact, as see last week, worst-case rutime is Θ( 2 ).

4 Approach There are may ways to desig algorithms. For example, isertio sort is icremetal: havig sorted A[1.. j 1], place A[j] correctly, so that A[1.. j] is sorted. is aother commo approach: Divide the problem ito a umber of subproblems that are smaller istaces of the same problem. the subproblems by solvig them recursively. Base case: If the subproblem are small eough, just solve them by brute force. Naive Mi-Max Fid miimum ad maximum of a list A of >0 umbers. Naive-Mi-Max(A) 1 least = A[1] 2 for i = 2 to A.legth 3 if A[i] < least 4 least = A[i] 5 greatest = A[1] 6 for i = 2 to A.legth 7 if A[i] > greatest 8 greatest = A[i] 9 retur (least, greatest) Combie the subproblem solutios to give a solutio to the origial problem. The for-loop o lie 2 makes 1 comparisos, as does the for-loop o lie 6, makig a total of 2 2 comparisos. Ca we do better? Yes! Mi-Max Mi-Max Algorithm Iitially called with Mi-Max(A, 1, A. legth). As we are dealig with subproblems, we state each subproblem as computig miimum ad maximum of a subarray A[p.. q]. Iitially, p = 1 ad q = A.legth, but these values chage as we recurse through subproblems. To compute miimum ad maximum of A[p.. q]: Divide by splittig ito two subarrays A[p.. r] ad A[r+1.. q], where r is the halfway poit of A[p.. q]. by recursively computig miimum ad maximum of the two subarrays A[p.. r] ad A[r+1.. q]. Mi-Max(A, p, q) 1 if p = q 2 retur (A[p], A[q]) 3 if p = q 1 4 if A[p] < A[q] 5 retur (A[p], A[q]) 6 else retur (A[q], A[p]) 7 r = (p+q)/2 8 (mi1, max1) = Mi-Max(A, p, r) 9 (mi2, max2) = Mi-Max(A, r+1, q) 10 retur ( mi(mi1, mi2), max(max1, max2) ) Combie by computig the overall miimum as the mi of the two recursively computed miima, similar for the overall maximum. Note I lie 7, r computes the halfway poit of A[p.. q]. = q p + 1 is the umber of elemets from which we compute the mi ad max.

5 Solvig the Mi-Max Recurrece Let T () be the umber of comparisos made by Mi-Max(A, p, q), where = q p+1 is the umber of elemets from which we compute the mi ad max. The T (1) = 0, T (2) = 1, ad for N > 2: Claim Proof. T () = T ( /2 ) + T ( /2 ) + 2. T () = for = 2k 2, i.e., powers of 2. The proof is by iductio o k (usig = 2 k ). Base case: true for k=1, as T (2 1 ) = 1 = Iductio step: assumig T (2 k ) = 3 2 2k 2, we get ( ) T (2 k+1 ) = 2T (2 k )+2 = k 2 +2 = 3 2 2k+1 2 Solvig the Mi-Max Recurrece (cot d) Some remarks: 1 If we replace lie 7 of the algorithm by r = p+1, the the resultig rutime T () satisfies T () = for all > 0. 2 For example, T (6) = 7 whereas T (6) = 8. 3 It ca be show that at least comparisos are ecessary i the worst case to fid the maximum ad miimum of umbers for ay compariso-based algorithm: this is thus a lower boud o the problem. 4 Hece this (last) algorithm is provably optimal. Big-Oh, Omega, Theta by examples = O()? YES = O( 2 )? YES = Ω()? YES = Ω( 2 )? NO = Θ()? YES = Θ( 2 )? NO 7 2 = O(3 )? YES 8 2 = Ω(3 )? NO Ufoldig the recursio for Mi-Max We have 0 if = 1 T () = 1 if = 2 T (. 2 ) + T ( 2 ) + 2 else 1 T (1) = 0 2 T (2) = 1 3 T (3) = T (2) + T (1) + 2 = = 3 4 T (4) = T (2) + T (2) + 2 = = 4 5 T (5) = T (3) + T (2) + 2 = = 6 6 T (6) = T (3) + T (3) + 2 = = 8 7 T (7) = T (4) + T (3) + 2 = = = O( 2 )? YES 8 T (8) = T (4) + T (4) + 2 = = = Θ( 2 )? YES 9 T (9) = T (5) + T (4) + 2 = = si() = O(1)? YES 10 T (10) = T (5) + T (5) + 2 = = 14. We cout 4 steps +1 ad 5 steps +2 we guess T () 3 2.

6 Fidig the best mi-max algorithm 1 As you ca see i the sectio o the mi-max problem, for some iput sizes we ca validate the guess T () Oe ca ow try to fid a precise geeral formula for T (), 3 However we see that we have T (6) = 8, while we ca hadle this case with 7 comparisos. So perhaps we ca fid a better algorithm? 4 Ad that is the case: 1 If is eve, fid the mi-max for the first two elemets usig 1 compariso; if is odd, fid the mi-max for the first elemet usig 0 comparisos. 2 Now iteratively fid the mi-max of the ext two elemets usig 1 compariso, ad compute the ew curret mi-max usig 2 further comparisos. Ad so o... This yields a algorithm usig precisely comparisos. Ad this is precisely optimal for all. We lear: Here divide-ad-coqueor provided a good steppig stoe to fid a really good algorithm.