Divide & Conquer. Divide-and-conquer algorithms. Conventional product of polynomials. Conventional product of polynomials.

Transcription

1 Divide-ad-coquer algorithms Divide & Coquer Strategy: Divide the problem ito smaller subproblems of the same type of problem Solve the subproblems recursively Combie the aswers to solve the origial problem Jordi Cortadella ad Jordi Petit Departmet of Computer Sciece The work is doe i three places: I partitioig the problem ito subproblems I solvig the basic cases at the tail of the recursio I mergig the aswers of the subproblems to obtai the solutio of the origial problem Covetioal product of polyomials Divide & Coquer Dept. CS, UPC Covetioal product of polyomials Example: fuctio PolyomialProduct(P, Q) // P ad Q are vectors of coefficiets. // Returs R = P Q. // degree(p) = size(p)-1, degree(q) = size(q)-1. // degree(r) = degree(p)+degree(q). R = vector with size(p)+size(q)-1 zeros; for each P i for each Q j R i+j = R i+j + P i Q j retur R Complexity aalysis: Multiplicatio of polyomials of degree : O Additio of polyomials of degree : O() Divide & Coquer Dept. CS, UPC 3 Divide & Coquer Dept. CS, UPC 4

2 Product of polyomials: Divide&Coquer Assume that we have two polyomials with coefficiets (degree 1) P: Q: 1 P L Q L / P R Q R 0 Product of complex umbers The product of two complex umbers requires four multiplicatios: a + bi c + di = ac bd + bc + ad i Carl Friedrich Gauss ( ) oticed that it ca be doe with just three: ac, bd ad a + b c + d bc + ad = a + b c + d ac bd Show later Divide & Coquer Dept. CS, UPC 5 Product of polyomials with Gauss s trick A similar observatio applies for polyomial multiplicatio. Divide & Coquer Dept. CS, UPC 6 Polyomial multiplicatio: recursive step P Q P L Q L P L Q R + P R Q L P L + P R Q L + Q R P R Q R Divide & Coquer Dept. CS, UPC 7 Divide & Coquer Dept. CS, UPC 8

3 Patter of recursive calls Complexity aalysis Type equatio here. / / / /4 /4 /4 /4 /4 /4 /4 /4 / Brachig factor: log levels The time spet at level k is 3 k O k = 3 For k = 0, rutime is O. k O() For k = log, rutime is O 3 log, which is equal to O log 3. The rutime per level icreases geometrically by a factor of 3/ per level. The sum of ay icreasig geometric series is, withi a costat factor, simply the last term of the series. Therefore, the complexity is O Divide & Coquer Dept. CS, UPC 9 A popular recursio tree Divide & Coquer Dept. CS, UPC 10 Examples Brachig factor: / / log levels 1 1 / /4 /4 / Algorithm Brach c Rutime equatio Power (x y ) 1 0 T y = T yτ + O(1) Biary search 1 0 T = T Τ + O(1) Merge sort 1 T = T Τ + O() Polyomial product 4 1 T = 4 T Τ + O() Polyomial product (Gauss) 3 1 T = 3 T Τ + O() Example: efficiet sortig algorithms. T = T + O Algorithms may differ o the amout of work doe at each level: O c Divide & Coquer Dept. CS, UPC 11 Divide & Coquer Dept. CS, UPC 1

4 Master theorem Typical patter for Divide&Coquer algorithms: Split the problem ito a subproblems of size /b Solve each subproblem recursively Combie the aswers i O( c ) time Size Size /b Ruig time: T = a T Τ b + O( c ) Master theorem: Size /b O c if c > log b a (a < b c ) T = O c log if c = log b a (a = b c ) O log b a if c < log b a (a > b c ) Master theorem: recursio tree Brachig factor a. Depth log b Size 1 Width a log b = log b a Divide & Coquer Dept. CS, UPC 13 Master theorem: proof For simplicity, assume is a power of b. The base case is reached after log b levels. The kth level of the tree has a k subproblems of size Τb k. The total work doe at level k is: a k O b k c = O c a b c k As k goes from 0 (the root) to log b (the leaves), these umbers form a geometric series with ratio aτb c. We eed to fid the sum of such a series. T = O c 1 + a b b a a3 + + c c b 3c + + alogb b (log b )c Divide & Coquer Dept. CS, UPC 14 Master theorem: proof Case aτb c < 1. Decreasig series. The sum is domiated by the first term (k = 0): O( c ). Case aτb c < 1. Icreasig series. The sum is domiated by the last term (k = log b ): c a b c log b = c a log b = a (log a )(log b a) = log b a b log b c = a log b = Case aτb c = 1. We have O log terms all equal to O( c ). Divide & Coquer Dept. CS, UPC 15 Divide & Coquer Dept. CS, UPC 16

5 Master theorem: examples Ruig time: T = a T Τb + O c O c T = O c log O log b a if a < b c if a = b c if a > b c Algorithm a c Rutime equatio Complexity Power (x y ) 1 0 T y = T yτ + O(1) Biary search 1 0 T = T Τ + O(1) Merge sort 1 T = T Τ + O() O(log y) O(log ) O( log ) Polyomial product 4 1 T = 4 T Τ + O() O( ) Polyomial product (Gauss) 3 1 T = 3 T Τ + O() O( log 3 ) b = for all the examples Divide & Coquer Dept. CS, UPC 17 Quick sort with Hugaria, folk dace Quick sort (Toy Hoare, 1959) Suppose that we kow a umber x such that oe-half of the elemets of a vector are greater tha or equal to x ad oe-half of the elemets are smaller tha x. Partitio the vector ito two equal parts ( 1 comparisos) Sort each part recursively Problem: we do ot kow x. The algorithm also works o matter which x we pick for the partitio. We call this umber the pivot. Observatio: the partitio may be ubalaced. Divide & Coquer Dept. CS, UPC 18 pivot Quick sort: example Partitio Qsort Qsort The key step of quick sort is the partitioig algorithm. Questio: how to fid a good pivot? Divide & Coquer Dept. CS, UPC 19 Divide & Coquer Dept. CS, UPC 0

6 Quick sort Quick sort: partitio fuctio Partitio(A, left, right) // A[left..right]: segmet to be sorted // Returs the middle of the partitio with // A[middle] = pivot // A[left..middle-1] pivot // A[middle+1..right] > pivot x = A[left]; // the pivot i = left; j = right; while i < j do while A[i] x ad i right do i = i+1; while A[j] > x ad j left do j = j-1; if i < j the swap(a[i], A[j]); swap(a[left], A[j]); retur j; Divide & Coquer Dept. CS, UPC 1 Quick sort partitio: example Divide & Coquer Dept. CS, UPC Quick sort: algorithm pivot fuctio Qsort(A, left, right) // A[left..right]: segmet to be sorted if left < right the mid = Partitio(A, left, right); Qsort(A, left, mid-1); Qsort(A, mid+1, right); middle Divide & Coquer Dept. CS, UPC 3 Divide & Coquer Dept. CS, UPC 4

7 Quick sort: Hoare s partitio Quick sort partitio: example fuctio HoarePartitio(A, left, right) // A[left..right]: segmet to be sorted. // Output: The left part has elemets tha the pivot. // The right part has elemets tha the pivot. // Returs the idex of the last elemet of the left part. x = A[left]; // the pivot i = left-1; j = right+1; while true do do i = i+1; while A[i] < x; do j = j-1; while A[j] > x; if i j the retur j; pivot First swap: 4 is a setiel for R; 6 is a setiel for L o eed to check for boudaries i j swap(a[i], A[j]); Admire a uique piece of art by Hoare: The first swap creates two setiels. After that, the algorithm flies j (middle) Divide & Coquer Dept. CS, UPC 5 Quick sort with Hoare s partitio Divide & Coquer Dept. CS, UPC 6 Quick sort: hybrid approach fuctio Qsort(A, left, right) // A[left..right]: segmet to be sorted if left < right the mid = HoarePartitio(A, left, right); Qsort(A, left, mid); Qsort(A, mid+1, right); fuctio Qsort(A, left, right) // A[left..right]: segmet to be sorted. // K is a break-eve size i which isertio sort is // more efficiet tha quick sort. if right left K the mid = HoarePartitio(A, left, right); Qsort(A, left, mid); Qsort(A, mid+1, right); fuctio Sort(A): Qsort(A, 0, A.size()-1); IsertioSort(A); Divide & Coquer Dept. CS, UPC 7 Divide & Coquer Dept. CS, UPC 8

8 Quick sort: complexity aalysis The partitio algorithm is O(). Assume that the partitio is balaced: T = T( Τ) + O = O( log ) Worst case rutime: the pivot is always the smallest elemet i the vector O( ) Selectig a good pivot is essetial. There are differet strategies, e.g., Take the media of the first, last ad middle elemets Take the pivot at radom Quick sort: complexity aalysis Let us assume that x i is the ith smallest elemet i the vector. Let us assume that each elemet has the same probability of beig selected as pivot. The rutime if x i is selected as pivot is: T = 1 + T i 1 + T( i) Divide & Coquer Dept. CS, UPC 9 Quick sort: complexity aalysis T = T i 1 + T( i) i=1 T = T i T( i) i=1 i=1 1 T = 1 + T i ( + 1)(H ) i=0 H = 1 + 1Τ + 1Τ Τ is the Harmoic series, that has a simple approximatio: H = l + γ + O 1Τ. γ = is Euler s costat. T + 1 l + γ O 1 = O( log ) Divide & Coquer Dept. CS, UPC 30 Quick sort: complexity aalysis summary Rutime of quicksort: T = O T = Ω( log ) T avg = O( log ) Be careful: some malicious patters may icrease the probability of the worst case rutime, e.g., whe the vector is sorted or almost sorted. Possible solutio: use radom pivots. Divide & Coquer Dept. CS, UPC 31 Divide & Coquer Dept. CS, UPC 3

9 The selectio problem Give a collectio of N elemets, fid the kth smallest elemet. Optios: Sort a vector ad select the kth locatio: O(N log N) Read k elemets ito a vector ad sort them. The remaiig elemets are processed oe by oe ad placed i the correct locatio (similar to isertio sort). Oly k elemets are maitaied i the vector. Complexity: O kn. Why? The selectio problem usig a heap Algorithm: Build a heap from the collectio of elemets: O(N) Remove k elemets: O(k log N) Note: heaps will be see later i the course Complexity: I geeral: O N + k log N For small values of k, i.e., k = O( NΤlog N), the complexity is O(N). For large values of k, the complexity is O(k log N). Divide & Coquer Dept. CS, UPC 33 Quick sort with Hoare s partitio Divide & Coquer Dept. CS, UPC 34 Quick select with Hoare s partitio fuctio Qsort(A, left, right) // A[left..right]: segmet to be sorted if left < right the mid = HoarePartitio(A, left, right); Qsort(A, left, mid); Qsort(A, mid+1, right); // Returs the elemet at locatio k assumig // A[left..right] would be sorted i ascedig order. // Pre: left k right. // Post: The elemets of A have chaged their locatios. fuctio Qselect(A, left, right, k) if left == right the retur A[left]; mid = HoarePartitio(A, left, right); // We oly eed to sort oe half of A if k mid the Qselect(A, left, mid, k); else Qselect(A, mid+1, right, k); Divide & Coquer Dept. CS, UPC 35 Divide & Coquer Dept. CS, UPC 36

10 Quick Select: complexity The Closest-Poits problem Assume that the partitio is balaced: Quick sort: T = T( Τ) + O = O( log ) Quick select: T = T( Τ) + O = O() Iput: A list of poits i the plae x 1, y 1, x, y,, x, y Output: The pair of closest poits Simple approach: check all pairs O( ) We wat a O( log ) solutio! The average liear time complexity ca be achieved by choosig good pivots (similar strategy ad complexity computatio to qsort). Divide & Coquer Dept. CS, UPC 37 The Closest-Poits problem We ca assume that the poits are sorted by the x-coordiate. Sortig the poits is free from the complexity stadpoit (O( log )). Split the list ito two halves. The closest poits ca be both at the left, both at the right or oe at the left ad the other at the right (ceter). The left ad right pairs are easy to fid (recursively). How about the pairs i the ceter? Divide & Coquer Dept. CS, UPC 38 The Closest-Poits problem Let δ = mi δ L, δ R. We oly eed to compute δ C if it improves δ. We ca defie a strip aroud de ceter with distace δ at the left ad right. If δ C improves δ, the the poits must be withi the strip. I the worst case, all poits ca still reside i the strip. But how may poits do we really have to cosider? δ L δ C δ L δ R δ R δ δ Divide & Coquer Dept. CS, UPC 39 Divide & Coquer Dept. CS, UPC 40

11 The Closest-Poits problem The Closest-Poits problem Let us take all poits i the strip ad sort them by the y-coordiate. We oly eed to cosider pairs of poits with distace smaller tha δ. Oce we fid a pair (p i, p j ) with y-coordiates that differ by more tha δ, we ca move to the ext p i. for (i=0; i < NumPoitsIStrip; ++i) for (j=i+1; j < NumPoitsIStrip; ++j) if (p i ad p j s y-coordiate differ by more tha δ) break; // Go to ext p i if (dist p i, p j < δ) δ = dist p i, p j ; For every poit p i at oe side of the strip, we oly eed to cosider poits from p i+1. The relevat poits oly reside i the δ δ rectagle below poit p i. There ca oly be 8 poits at most i this rectagle (4 at the left ad 4 at the right). Some poits may have the same coordiates. δ δ δ But, how may pairs p i, p j do we eed to cosider? Divide & Coquer Dept. CS, UPC 41 The Closest-Poits problem: algorithm Sort the poits accordig to their x-coordiates. Divide the set ito two equal-sized parts. Compute the mi distace at each part (recursively). Let δ be the miimal of the two miimal distaces. Elimiate poits that are farther tha δ from the separatio lie. Sort the remaiig poits accordig to their y-coordiates. Sca the remaiig poits i the y order ad compute the distaces of each poit to its 7 eighbors. Divide & Coquer Dept. CS, UPC 4 The Closest-Poits problem: complexity Iitial sort usig x-coordiates: O( log ). It comes for free. Divide ad coquer: Solve for each part recursively: T Τ Elimiate poits farther tha δ: O Sort remaiig poits usig y-coordiates: O log Sca the remaiig poits i y order: O T = T Τ + O + O log = O( log ) Ca we do it i O( log )? Yes, we eed to sort by y i a smart way. Divide & Coquer Dept. CS, UPC 43 Divide & Coquer Dept. CS, UPC 44

12 The Closest-Poits problem: complexity Let Y a vector with the poits sorted by the y-coordiates. This ca be doe iitially for free. Each time we partitio the set of poits by the x-coordiate, we also partitio Y ito two sorted vectors (usig a umergig procedure with liear complexity) Y L = Y R = // Iitial lists of poits foreach p i Y i ascedig order of y do if p i is at the left part the Y L.push_back(p i ) else Y R.push_back(p i ) APPENDIX Now, sortig the poits by the y-coordiate at each iteratio ca be doe i liear time, ad the problem ca be solved i O( log ) Divide & Coquer Dept. CS, UPC 45 T = 1 + i=0 Full-history recurrece relatio 1 T i 1 T = 1 + T i, i=0 + 1 T + 1 = T(i) + 1 T + 1 T = T = + T() T + 1 = + T T T A recurrece that depeds o all the previous values of the fuctio. T i=0 Divide & Coquer Dept. CS, UPC 46 Harmoic series H = k = k=1 k න 1 k= +1 dx l( + 1) 1 x k k=1 T = ( + 1) H = Θ(log ) T + 1 H = Θ( log ) Divide & Coquer Dept. CS, UPC 47 Divide & Coquer Dept. CS, UPC 48