Chapter 5. Divide and Conquer CLRS 4.3. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
|
|
- Kerrie Cunningham
- 5 years ago
- Views:
Transcription
1 Chapter 5 Divide and Conquer CLRS 4.3 Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. 1
2 Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. Most common usage. Break up problem of size n into two equal parts of size ½n. Solve two parts recursively. Combine two solutions into overall solution in linear time. Consequence. Brute force: n 2. Divide-and-conquer: n log n. Divide et impera. Veni, vidi, vici. - Julius Caesar 2
3 5.1 Mergesort
4 Sorting Sorting. Given n elements, rearrange in ascending order. Obvious sorting applications. List files in a directory. Organize an MP3 library. List names in a phone book. Display Google PageRank results. Problems become easier once sorted. Find the median. Find the closest pair. Binary search in a database. Identify statistical outliers. Find duplicates in a mailing list. Non-obvious sorting applications. Data compression. Computer graphics. Interval scheduling. Computational biology. Minimum spanning tree. Supply chain management. Simulate a system of particles. Book recommendations on Amazon. Load balancing on a parallel computer.... 4
5 Mergesort Mergesort. Divide array into two halves. Recursively sort each half. Merge two halves to make sorted whole. Jon von Neumann (1945) A L G O R I T H M S A L G O R I T H M S divide O(1) A G L O R H I M S T sort 2T(n/2) A G H I L M O R S T merge O(n) 5
6 Merging Merging. Combine two pre-sorted lists into a sorted whole. How to merge efficiently? Linear number of comparisons. Use temporary array. A G L O R H I M S T A G H I Challenge for the bored. In-place merge. [Kronrod, 1969] using only a constant amount of extra storage 6
7 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A auxiliary array 7
8 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A G auxiliary array 8
9 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A G H auxiliary array 9
10 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A G H I auxiliary array 10
11 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A G H I L auxiliary array 11
12 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A G H I L M auxiliary array 12
13 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A G H I L M O auxiliary array 13
14 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. smallest smallest A G L O R H I M S T A G H I L M O R auxiliary array 14
15 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. first half exhausted smallest A G L O R H I M S T A G H I L M O R S auxiliary array 15
16 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. first half exhausted smallest A G L O R H I M S T A G H I L M O R S T auxiliary array 16
17 Merging Merge. Keep track of smallest element in each sorted half. Insert smallest of two elements into auxiliary array. Repeat until done. first half exhausted second half exhausted A G L O R H I M S T A G H I L M O R S T auxiliary array 17
18 A Useful Recurrence Relation Def. T(n) = number of comparisons to mergesort an input of size n. Mergesort recurrence. Solution. T(n) O(n log 2 n). Assorted proofs. We describe several ways to prove this recurrence. Initially we assume n is a power of 2 and replace with =. 18
19 Proof by Recursion Tree T(n) n T(n/2) T(n/2) 2(n/2) T(n/4) T(n/4) T(n/4) T(n/4) log 2 n 4(n/4)... T(n / 2 k ) 2 k (n / 2 k )... T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) n/2 (2) n log 2 n 19
20 Proof by Induction Claim. If T(n) satisfies this recurrence, then T(n) = n log 2 n. assumes n is a power of 2 Pf. (by induction on n) Base case: n = 1. Inductive hypothesis: T(n) = n log 2 n. Goal: show that T(2n) = 2n log 2 (2n). 20
21 Analysis of Mergesort Recurrence Claim. If T(n) satisfies the following recurrence, then T(n) n lg n. log 2 n Pf. (by induction on n) Base case: n = 1. T(1) = 0 = 1 lg 1. Define n 1 = n / 2, n 2 = n / 2. Induction step: Let n 2, assume true for 1, 2,..., n 1. 21
22 5.4 Closest Pair of Points
23 Closest Pair of Points Closest pair. Given n points in the plane, find a pair with smallest Euclidean distance between them. Fundamental geometric primitive. Graphics, computer vision, geographic information systems, molecular modeling, air traffic control. Special case of nearest neighbor, Euclidean MST, Voronoi. fast closest pair inspired fast algorithms for these problems Brute force. Check all pairs of points p and q with Θ(n 2 ) comparisons. 1-D version. O(n log n) easy if points are on a line. Assumption. No two points have same x coordinate. to make presentation cleaner 23
24 Closest Pair of Points Algorithm. Divide: draw vertical line L so that roughly ½n points on each side. L 24
25 Closest Pair of Points Algorithm. Divide: draw vertical line L so that roughly ½n points on each side. Conquer: find closest pair in each side recursively. L
26 Closest Pair of Points Algorithm. Divide: draw vertical line L so that roughly ½n points on each side. Conquer: find closest pair in each side recursively. Combine: find closest pair with one point in each side. seems like Θ(n 2 ) Return best of 3 solutions. L
27 Closest Pair of Points Find closest pair with one point in each side, assuming that distance < δ. L δ = min(12, 21) 27
28 Closest Pair of Points Find closest pair with one point in each side, assuming that distance < δ. Observation: only need to consider points within δ of line L. L δ = min(12, 21) δ 28
29 Closest Pair of Points Find closest pair with one point in each side, assuming that distance < δ. Observation: only need to consider points within δ of line L. Sort points in 2δ-strip by their y coordinate. 7 L δ = min(12, 21) 2 1 δ 29
30 Closest Pair of Points Find closest pair with one point in each side, assuming that distance < δ. Observation: only need to consider points within δ of line L. Sort points in 2δ-strip by their y coordinate. Only check distances of those within 11 positions in sorted list! 7 L δ = min(12, 21) 2 1 δ 30
31 Closest Pair of Points Def. Let s i be the point in the 2δ-strip, with the i th smallest y-coordinate. Claim. If i j 12, then the distance between s i and s j is at least δ. Pf j No two points lie in same ½δ-by-½δ box. Two points at least 2 rows apart have distance 2(½δ) = δ. 2 rows ½δ ½δ Fact. Still true if we replace 12 with 7. i ½δ Scan points in y-order and compare distance between each point and next 11 neighbours. If any of these distances is less than δ, update δ. δ δ 31
32 Closest Pair of Points Smallest-Dist(p 1,, p n ) { if n=2 then return dist(p 1,p 2 ) Compute separation line L such that half the points are on one side and half on the other side. δ = Smallest-Dist(left half) δ = Smallest-Dist(right half) δ = min(δ,δ ) Delete all points further than δ from separation line L Sort remaining points by y-coordinate. Scan points in y-order and compare distance between each point and next 11 neighbours. If any of these distances is less than δ, update δ. O(n log n) 2T(n / 2) O(n) O(n log n) O(n) } return δ. 32
33 Closest Pair of Points Closest-Pair(p 1,, p n ) { if n=2 then return dist(p 1,p 2 ),p 1,p 2 Compute separation line L such that half the points are on one side and half on the other side. δ,p,q = Closest-Pair(left half) δ,p,q = Closest-Pair(right half) δ, p, q = min(δ, δ )(p,q,p,q ) Delete all points further than δ from separation line L Sort remaining points by y-coordinate. Scan points in y-order and compare distance between each point and next 11 neighbours. If any of these distances is less than δ, update δ,p,q. O(n log n) 2T(n / 2) O(n) O(n log n) O(n) } return δ,p,q. 33
34 Closest Pair of Points: Analysis Running time. Q. Can we achieve O(n log n)? A. Yes. First sort all points according to x coordinate before algo. Don t sort points in strip from scratch each time. Each recursion returns a list: all points sorted by y coordinate. Sort by merging two pre-sorted lists. 34
35 Matrix Multiplication
36 Matrix Multiplication Matrix multiplication. Given two n-by-n matrices A and B, compute C = AB. Brute force. Θ(n 3 ) arithmetic operations. Fundamental question. Can we improve upon brute force? 36
37 Matrix Multiplication: Warmup Divide-and-conquer. Divide: partition A and B into ½n-by-½n blocks. Conquer: multiply 8 ½n-by-½n recursively. Combine: add appropriate products using 4 matrix additions. 37
38 Matrix Multiplication: Key Idea Key idea. multiply 2-by-2 block matrices with only 7 multiplications. 7 multiplications. 18 = additions (or subtractions). 38
39 Strassen: Recursion Tree 7 n 2 n 2 =n T(n) n 2 T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) T(n/2) 7(n 2 /2 2 ) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4)... T(n/4) T(n/4) T(n/4) T(n/4) log 2 n 49(n 2 /4 2 )... T(n / 2 k ) 7 k (n 2 / 4 k ) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) 7 lg n (n 2 / 4 lg n ) =7 lg n = n lg 7 39
40 Fast Matrix Multiplication Fast matrix multiplication. (Strassen, 1969) Divide: partition A and B into ½n-by-½n blocks. Compute: 14 ½n-by-½n matrices via 10 matrix additions. Conquer: multiply 7 ½n-by-½n matrices recursively. Combine: 7 products into 4 terms using 18 matrix additions. Analysis. Assume n is a power of 2. T(n) = # arithmetic operations. 40
41 Fast Matrix Multiplication in Practice Implementation issues. Sparsity. Caching effects. Numerical stability. Odd matrix dimensions. Crossover to classical algorithm around n = 128. Common misperception: "Strassen is only a theoretical curiosity." Advanced Computation Group at Apple Computer reports 8x speedup on G4 Velocity Engine when n ~ 2,500. Range of instances where it's useful is a subject of controversy. Remark. Can "Strassenize" Ax=b, determinant, eigenvalues, and other matrix ops. 41
42 Fast Matrix Multiplication in Theory Q. Multiply two 2-by-2 matrices with only 7 scalar multiplications? A. Yes! [Strassen, 1969] Θ(n log 2 7 ) = O(n 2.81 ) Q. Multiply two 2-by-2 matrices with only 6 scalar multiplications? A. Impossible. [Hopcroft and Kerr, 1971] (n log 2 Θ 6 ) = O(n 2.59 ) Q. Two 3-by-3 matrices with only 21 scalar multiplications? (n log 3 A. Also impossible. Θ 21 ) = O(n 2.77 ) Q. Two 70-by-70 matrices with only 143,640 scalar multiplications? A. Yes! [Pan, 1980] (n log 70 Θ ) = O(n 2.80 ) Decimal wars. December, 1979: O(n ). January, 1980: O(n ). 42
43 Fast Matrix Multiplication in Theory Best known. O(n ) [Coppersmith-Winograd, ] In 2010, Andrew Stothers gave an improvement to the algorithm O(n ). In 2011, Virginia Williams combined a mathematical short-cut from Stothers' paper with her own insights and automated optimization on computers, improving the bound O(n ). In 2014, François Le Gall simplified the methods of Williams and obtained an improved bound of O(n ). Conjecture. O(n 2+ε ) for any ε > 0. Caveat. Theoretical improvements to Strassen are progressively less practical (hidden constant gets worse). 43
44 CLRS 4.3 Master Theorem
45 Master Theorem from CLRS 4.3 T(n) = at(n/b) + f(n) a = (constant) number of sub-instances, b = (constant) size ratio of sub-instances, f(n) = time used for dividing and recombining. 45
46 Proof by Recursion Tree T(n) = at(n/b) + f(n) T(n) f(n) T(n/b) T(n/b) T(n/b) af(n/b) T(n/b 2 ) T(n/b 2 ) T(n/b 2 ) T(n/b 2 ) T(n/b 2 ) T(n/b 2 ) T(n/b 2 ) T(n/b 2 ) T(n/b 2 ) a 2 f(n/b 2 )... log b n... T(n / b k ) T(n / b k ) T(n / b k ) T(n / b k ) T(n / b k )... T(n / b k ) T(n / b k ) T(n / b k ) T(n / b k ) a k f(n/b k )... T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) a log b n T(2) = T(2) n log b a T(n) = a k f(n/b k ) 46
47 T(n) = at(n/b) + f(n) Case 1: f(n) O(n L ) for some constant L < logb a. Solution: T(n) Θ(n log b a ) Case 2: f(n) Θ(n log b a log k n), for some k 0. Solution: T(n) Θ(n log b a log k+1 n) Case 3: f(n) is Ω(n L ) for some constant L > logb a and f(n) satisfies the regularity condition af(n/b) cf(n) for some c<1 and all large n. Solution: T(n) Θ(f(n)) 47
48 Master Theorem T(n) = at(n/b) + f(n) Case 1: f(n) O(n L ) for some constant L < logb a. is Solution: T(n) Θ(n log b a ) 48
49 Master Theorem Case 1: f(n) O(n L ) for some constant L < logb a. Solution: T(n) Θ(n log b a ) T(n) = 5T(n/2) + Θ(n 2 ) Compare n log 2 5 vs. n 2. Since 2 < log2 5 use Case 1 Solution: T(n) Θ(n log 2 5 ) 49
50 Master Theorem T(n) = at(n/b) + f(n) Simple Case 2: f(n) Θ(n log b a ). Solution: T(n) Θ(n log b a log n) 50
51 Master Theorem T(n) = at(n/b) + f(n) Case 2: f(n) Θ(n log b a log k n), for some k 0. Solution: T(n) Θ(n log b a log k+1 n) 51
52 Master Theorem Case 2: f(n) Θ(n log b a log k n), for some k 0. Solution: T(n) Θ(n log b a log k+1 n) T(n) = 27T(n/3) + Θ(n 3 log n) Compare n log 3 27 vs. n 3. Since 3 = log3 27 use Case 2 Solution: T(n) Θ(n 3 log 2 n) 52
53 Master Theorem T(n) = at(n/b) + f(n) Case 3: f(n) Ω(n L ) for some constant L > logb a and f(n) satisfies the regularity condition af(n/b) cf(n) for some c<1 and all large n. Solution: T(n) Θ(f(n)) 53
54 Master Theorem Case 3: f(n) Ω(n L ) for some constant L > logb a and f(n) satisfies the regularity condition af(n/b) cf(n) for some c<1 and all large n. Solution: T(n) Θ(f(n)) 54
55 Master Theorem Case 3: f(n) Ω(n L ) for some constant L > logb a and f(n) satisfies the regularity condition af(n/b) cf(n) for some c<1 and all large n. Solution: T(n) Θ(f(n)) T(n) = 5T(n/2) + Θ(n 3 ) Compare n log 2 5 vs. n 3. Since 3 > log2 5 use Case 3 af(n/b) = 5(n/2) 3 = 5/8 n 3 cn 3, for c = 5/8 Solution: T(n) Θ(n 3 ) 55
56 Master Theorem T(n) = 27T(n/3) + Θ(n 3 /log n) Compare n log 3 27 vs. n 3. Since 3 = log3 27 use Case 2 but n 3 /log n not Θ(n 3 log k n) for k 0 Cannot use Master Method. 56
57 Beyond the Master Theorem
58 Median Finding
59 Median Finding Median Finding. Given n distinct numbers a 1,, a n, find i such that { j : a j <a i } = n-1 / 2 and { j : a j >a i } = n-1 /
60 Selection Selection. Given n distinct numbers a 1,, a n, and index k, find i such that k=4 { j : a j <a i } = k-1 and { j : a j >a i } = n-k Median( a 1,, a n ) = Selection( a 1,, a n, n+1 / 2 ) 60
61 Partition (from QuickSort) Algorithm partition(a, start, stop) Input: An array A, indices start and stop. Output: Returns an index j and rearranges the elements of A such that for all i<j, A[i]! A[j] and for all k>i, j A[k] " A[j]. pivot # A[stop] left # start right # stop - 1 while left! right do while left! right and A[left]! pivot) do left # left + 1 while (left! right and A[right] " pivot) do right # right -1 if (left < right ) then exchange A[left] $ A[right] exchange A[stop] $ A[left] return left
62 Partition Example of execution of partition A = [ ] pivot = 5 A = [ ] swap 6, 2 A = [ ] A = [ ] swap 7,3 A = [ ] A = [ ] swap 7,pivot A = [ ]!5 "5
63 Selection from Median Selection. Given n distinct numbers a 1,, a n, and index k, find i such that { j : a j <a i } = k-1 and { j : a j >a i } = n-k. Median Partition M k = n+1 / 2 return k k k-m Selection( a 1,, a n-1 / 2,k ) Selection( a n+3 / 2,, a n, k - n+1 / 2 ) 63
64 Selection Selection. Given n distinct numbers a 1,, a n, and index k, find i such that { j : a j <a i } = k-1 and { j : a j >a i } = n-k. m Pseudo- Median Partition k = m return k Selection( a 1,, a m-1,k ) Selection( a m+1,, a n, k - m ) 64
65 Selection Selection. Given n distinct numbers a 1,, a n, and index k, find i such that { j : a j <a i } = k-1 and { j : a j >a i } = n-k. 3n/10 7n/10 65
66 Selection Selection. Given n distinct numbers a 1,, a n, and index k, find i such that { j : a j <a i } = k-1 and { j : a j >a i } = n-k. T(n) T( n/5 ) + Max{ T( 3n/10 T( 7n/10 ) } + Θ(n) Solution: T(n) Θ(n) 66
67 Selection Selection. Given n distinct numbers a 1,, a n, and index k, find i such that { j : a j <a i } = k-1 and { j : a j >a i } = n-k. T(n) T( n/5 ) + T( 7n/10 ) + Θ(n) Solution: T(n) Θ(n) Assuming T(i) d i for 1 i n, Θ(n) cn T(n+1) T(n+1 /5) + T(7(n+1)/10) + c(n+1) d(n+1)/5 + 7d(n+1)/10 + c(n+1) = (2d+7d+10c)/10 (n+1) = (9d+10c)/10 (n+1) d (n+1) as long as (9d+10c)/10 d, or equivalently 10c d. 67
68 Selection Selection. Given n distinct numbers a 1,, a n, and index k, find i such that { j : a j <a i } = k-1 and { j : a j >a i } = n-k. T(n) T( n/5 ) + T( 7n/10 ) + Θ(n) Solution: T(n) Θ(n) example: d=10c, Assuming T(i) 10c i for 1 i n, Θ(n) cn T(n+1) T(n+1 /5) + T(7/10 (n+1)) + c(n+1) 10c/5 (n+1) c/10 (n+1) + c(n+1) = (2c+7c+c)(n+1) = 10c (n+1) 68
69 Chapter 5 Divide and Conquer CLRS 4.3 Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. 69
Chapter 5. Divide and Conquer CLRS 4.3. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer CLRS 4.3 Slides by Kevin Wayne. Copyright 25 Pearson-Addison Wesley. All rights reserved. Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve
More information! Break up problem into several parts. ! Solve each part recursively. ! Combine solutions to sub-problems into overall solution.
Divide-and-Conquer Chapter 5 Divide and Conquer Divide-and-conquer.! Break up problem into several parts.! Solve each part recursively.! Combine solutions to sub-problems into overall solution. Most common
More informationCopyright 2000, Kevin Wayne 1
Divide-and-Conquer Chapter 5 Divide and Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. Most common
More informationDivide-and-Conquer. Consequence. Brute force: n 2. Divide-and-conquer: n log n. Divide et impera. Veni, vidi, vici.
Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. Most common usage. Break up problem of
More informationChapter 5. Divide and Conquer. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. 1 Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each
More informationMatrix Multiplication
Matrix Multiplication Matrix Multiplication Matrix multiplication. Given two n-by-n matrices A and B, compute C = AB. n c ij = a ik b kj k=1 c 11 c 12 c 1n c 21 c 22 c 2n c n1 c n2 c nn = a 11 a 12 a 1n
More information5. DIVIDE AND CONQUER I
5. DIVIDE AND CONQUER I mergesort counting inversions closest pair of points randomized quicksort median and selection Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More information5. DIVIDE AND CONQUER I
5. DIVIDE AND CONQUER I mergesort counting inversions closest pair of points median and selection Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationAlgorithm Design and Analysis
Algorithm Design and Analysis LECTURE 9 Divide and Conquer Merge sort Counting Inversions Binary Search Exponentiation Solving Recurrences Recursion Tree Method Master Theorem Sofya Raskhodnikova S. Raskhodnikova;
More informationDivide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch] Divide and Conquer Paradigm An important general technique for designing algorithms: divide problem into subproblems recursively
More informationDivide-Conquer-Glue Algorithms
Divide-Conquer-Glue Algorithms Mergesort and Counting Inversions Tyler Moore CSE 3353, SMU, Dallas, TX Lecture 10 Divide-and-conquer. Divide up problem into several subproblems. Solve each subproblem recursively.
More informationChapter 5. Divide and Conquer. CS 350 Winter 2018
Chapter 5 Divide and Conquer CS 350 Winter 2018 1 Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve each part recursively. Combine solutions to sub-problems into overall
More informationDivide-and-conquer: Order Statistics. Curs: Fall 2017
Divide-and-conquer: Order Statistics Curs: Fall 2017 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers.
More informationDivide-and-conquer. Curs 2015
Divide-and-conquer Curs 2015 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers. Known Examples: Binary
More informationCSE 421 Algorithms: Divide and Conquer
CSE 42 Algorithms: Divide and Conquer Larry Ruzzo Thanks to Richard Anderson, Paul Beame, Kevin Wayne for some slides Outline: General Idea algorithm design paradigms: divide and conquer Review of Merge
More informationCPS 616 DIVIDE-AND-CONQUER 6-1
CPS 616 DIVIDE-AND-CONQUER 6-1 DIVIDE-AND-CONQUER Approach 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger)
More informationChapter 2. Recurrence Relations. Divide and Conquer. Divide and Conquer Strategy. Another Example: Merge Sort. Merge Sort Example. Merge Sort Example
Recurrence Relations Chapter 2 Divide and Conquer Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative
More informationDivide and Conquer Strategy
Divide and Conquer Strategy Algorithm design is more an art, less so a science. There are a few useful strategies, but no guarantee to succeed. We will discuss: Divide and Conquer, Greedy, Dynamic Programming.
More informationDivide and Conquer. Andreas Klappenecker
Divide and Conquer Andreas Klappenecker The Divide and Conquer Paradigm The divide and conquer paradigm is important general technique for designing algorithms. In general, it follows the steps: - divide
More informationLinear Time Selection
Linear Time Selection Given (x,y coordinates of N houses, where should you build road These lecture slides are adapted from CLRS.. Princeton University COS Theory of Algorithms Spring 0 Kevin Wayne Given
More informationHow to Multiply. 5.5 Integer Multiplication. Complex Multiplication. Integer Arithmetic. Complex multiplication. (a + bi) (c + di) = x + yi.
How to ultiply Slides by Kevin Wayne. Copyright 5 Pearson-Addison Wesley. All rights reserved. integers, matrices, and polynomials Complex ultiplication Complex multiplication. a + bi) c + di) = x + yi.
More informationDivide and Conquer. Recurrence Relations
Divide and Conquer Recurrence Relations Divide-and-Conquer Strategy: Break up problem into parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. 2 MergeSort Mergesort.
More informationCS 5321: Advanced Algorithms - Recurrence. Acknowledgement. Outline. Ali Ebnenasir Department of Computer Science Michigan Technological University
CS 5321: Advanced Algorithms - Recurrence Ali Ebnenasir Department of Computer Science Michigan Technological University Acknowledgement Eric Torng Moon Jung Chung Charles Ofria Outline Motivating example:
More informationData Structures and Algorithms Chapter 3
Data Structures and Algorithms Chapter 3 1. Divide and conquer 2. Merge sort, repeated substitutions 3. Tiling 4. Recurrences Recurrences Running times of algorithms with recursive calls can be described
More informationCS 5321: Advanced Algorithms Analysis Using Recurrence. Acknowledgement. Outline
CS 5321: Advanced Algorithms Analysis Using Recurrence Ali Ebnenasir Department of Computer Science Michigan Technological University Acknowledgement Eric Torng Moon Jung Chung Charles Ofria Outline Motivating
More information5. DIVIDE AND CONQUER I
5. DIVIDE AND CONQUER I mergesort counting inversions randomized quicksort median and selection closest pair of points Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationDIVIDE AND CONQUER II
DIVIDE AND CONQUER II master theorem integer multiplication matrix multiplication convolution and FFT Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More information1. Basic Algorithms: Bubble Sort
Sorting Algorithms Sorting Sorting. Given n elements, rearrange in ascending order. Obvious sorting applications. List files in a directory. Organize an MP3 library. List names in a phone book. Display
More informationCopyright 2000, Kevin Wayne 1
//8 Fast Integer Division Too (!) Schönhage Strassen algorithm CS 8: Algorithm Design and Analysis Integer division. Given two n-bit (or less) integers s and t, compute quotient q = s / t and remainder
More informationAnalysis of Algorithms - Using Asymptotic Bounds -
Analysis of Algorithms - Using Asymptotic Bounds - Andreas Ermedahl MRTC (Mälardalens Real-Time Research Center) andreas.ermedahl@mdh.se Autumn 004 Rehersal: Asymptotic bounds Gives running time bounds
More information1. Basic Algorithms: Bubble Sort
Sorting Algorithms Sorting Sorting. Given n elements, rearrange in ascending order. Obvious sorting applications. List files in a directory. Organize an MP3 library. List names in a phone book. Display
More informationData Structures and Algorithms Chapter 3
1 Data Structures and Algorithms Chapter 3 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples
More informationCS 580: Algorithm Design and Analysis
CS 580: Algorithm Design and Analysis Jeremiah Blocki Purdue University Spring 208 Announcement: Homework 3 due February 5 th at :59PM Final Exam (Tentative): Thursday, May 3 @ 8AM (PHYS 203) Recap: Divide
More informationAlgorithms, Design and Analysis. Order of growth. Table 2.1. Big-oh. Asymptotic growth rate. Types of formulas for basic operation count
Types of formulas for basic operation count Exact formula e.g., C(n) = n(n-1)/2 Algorithms, Design and Analysis Big-Oh analysis, Brute Force, Divide and conquer intro Formula indicating order of growth
More informationDivide & Conquer. Jordi Cortadella and Jordi Petit Department of Computer Science
Divide & Conquer Jordi Cortadella and Jordi Petit Department of Computer Science Divide-and-conquer algorithms Strategy: Divide the problem into smaller subproblems of the same type of problem Solve the
More informationLecture 4. Quicksort
Lecture 4. Quicksort T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018 Networking
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 4: Divide and Conquer (I) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ Divide and Conquer ( DQ ) First paradigm or framework DQ(S)
More informationThe Divide-and-Conquer Design Paradigm
CS473- Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm CS473 Lecture 4 1 The Divide-and-Conquer Design Paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems
More informationCMPS 2200 Fall Divide-and-Conquer. Carola Wenk. Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk
CMPS 2200 Fall 2017 Divide-and-Conquer Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 1 The divide-and-conquer design paradigm 1. Divide the problem (instance)
More informationAsymptotic Analysis and Recurrences
Appendix A Asymptotic Analysis and Recurrences A.1 Overview We discuss the notion of asymptotic analysis and introduce O, Ω, Θ, and o notation. We then turn to the topic of recurrences, discussing several
More informationChapter 4 Divide-and-Conquer
Chapter 4 Divide-and-Conquer 1 About this lecture (1) Recall the divide-and-conquer paradigm, which we used for merge sort: Divide the problem into a number of subproblems that are smaller instances of
More informationReview Of Topics. Review: Induction
Review Of Topics Asymptotic notation Solving recurrences Sorting algorithms Insertion sort Merge sort Heap sort Quick sort Counting sort Radix sort Medians/order statistics Randomized algorithm Worst-case
More informationCSCI 3110 Assignment 6 Solutions
CSCI 3110 Assignment 6 Solutions December 5, 2012 2.4 (4 pts) Suppose you are choosing between the following three algorithms: 1. Algorithm A solves problems by dividing them into five subproblems of half
More informationAlgorithms. Quicksort. Slide credit: David Luebke (Virginia)
1 Algorithms Quicksort Slide credit: David Luebke (Virginia) Sorting revisited We have seen algorithms for sorting: INSERTION-SORT, MERGESORT More generally: given a sequence of items Each item has a characteristic
More informationData Structures and Algorithms CSE 465
Data Structures and Algorithms CSE 465 LECTURE 8 Analyzing Quick Sort Sofya Raskhodnikova and Adam Smith Reminder: QuickSort Quicksort an n-element array: 1. Divide: Partition the array around a pivot
More informationLecture 1: Asymptotics, Recurrences, Elementary Sorting
Lecture 1: Asymptotics, Recurrences, Elementary Sorting Instructor: Outline 1 Introduction to Asymptotic Analysis Rate of growth of functions Comparing and bounding functions: O, Θ, Ω Specifying running
More informationAlgorithms And Programming I. Lecture 5 Quicksort
Algorithms And Programming I Lecture 5 Quicksort Quick Sort Partition set into two using randomly chosen pivot 88 31 25 52 14 98 62 30 23 79 14 31 2530 23 52 88 62 98 79 Quick Sort 14 31 2530 23 52 88
More informationCS/COE 1501 cs.pitt.edu/~bill/1501/ Integer Multiplication
CS/COE 1501 cs.pitt.edu/~bill/1501/ Integer Multiplication Integer multiplication Say we have 5 baskets with 8 apples in each How do we determine how many apples we have? Count them all? That would take
More informationCMPSCI611: Three Divide-and-Conquer Examples Lecture 2
CMPSCI611: Three Divide-and-Conquer Examples Lecture 2 Last lecture we presented and analyzed Mergesort, a simple divide-and-conquer algorithm. We then stated and proved the Master Theorem, which gives
More informationAlgorithms Test 1. Question 1. (10 points) for (i = 1; i <= n; i++) { j = 1; while (j < n) {
Question 1. (10 points) for (i = 1; i
More informationCS 577 Introduction to Algorithms: Strassen s Algorithm and the Master Theorem
CS 577 Introduction to Algorithms: Jin-Yi Cai University of Wisconsin Madison In the last class, we described InsertionSort and showed that its worst-case running time is Θ(n 2 ). Check Figure 2.2 for
More informationThe maximum-subarray problem. Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm:
The maximum-subarray problem Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm: Brute force algorithm: At best, θ(n 2 ) time complexity 129 Can we do divide
More informationCOMP 250: Quicksort. Carlos G. Oliver. February 13, Carlos G. Oliver COMP 250: Quicksort February 13, / 21
COMP 250: Quicksort Carlos G. Oliver February 13, 2018 Carlos G. Oliver COMP 250: Quicksort February 13, 2018 1 / 21 1 1 https://xkcd.com/1667/ Carlos G. Oliver COMP 250: Quicksort February 13, 2018 2
More informationb + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d
CS161, Lecture 4 Median, Selection, and the Substitution Method Scribe: Albert Chen and Juliana Cook (2015), Sam Kim (2016), Gregory Valiant (2017) Date: January 23, 2017 1 Introduction Last lecture, we
More informationDivide and Conquer Algorithms
Divide and Conquer Algorithms Introduction There exist many problems that can be solved using a divide-and-conquer algorithm. A divide-andconquer algorithm A follows these general guidelines. Divide Algorithm
More informationMethods for solving recurrences
Methods for solving recurrences Analyzing the complexity of mergesort The merge function Consider the following implementation: 1 int merge ( int v1, int n1, int v, int n ) { 3 int r = malloc ( ( n1+n
More informationAnalysis of Algorithms CMPSC 565
Analysis of Algorithms CMPSC 565 LECTURES 38-39 Randomized Algorithms II Quickselect Quicksort Running time Adam Smith L1.1 Types of randomized analysis Average-case analysis : Assume data is distributed
More informationA design paradigm. Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/ EECS 3101
A design paradigm Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/17 112 Multiplying complex numbers (from Jeff Edmonds slides) INPUT: Two pairs of integers, (a,b),
More informationAlgorithms and Data Structures Strassen s Algorithm. ADS (2017/18) Lecture 4 slide 1
Algorithms and Data Structures Strassen s Algorithm ADS (2017/18) Lecture 4 slide 1 Tutorials Start in week (week 3) Tutorial allocations are linked from the course webpage http://www.inf.ed.ac.uk/teaching/courses/ads/
More informationQuicksort (CLRS 7) We previously saw how the divide-and-conquer technique can be used to design sorting algorithm Merge-sort
Quicksort (CLRS 7) We previously saw how the divide-and-conquer technique can be used to design sorting algorithm Merge-sort Partition n elements array A into two subarrays of n/2 elements each Sort the
More informationDivide-and-Conquer. a technique for designing algorithms
Divide-and-Conquer a technique for designing algorithms decomposing instance to be solved into subinstances of the same problem solving each subinstance combining subsolutions to obtain the solution to
More informationDivide & Conquer. Jordi Cortadella and Jordi Petit Department of Computer Science
Divide & Conquer Jordi Cortadella and Jordi Petit Department of Computer Science Divide-and-conquer algorithms Strategy: Divide the problem into smaller subproblems of the same type of problem Solve the
More informationAlgorithms. Algorithms 2.2 MERGESORT. mergesort bottom-up mergesort sorting complexity divide-and-conquer ROBERT SEDGEWICK KEVIN WAYNE
Algorithms ROBERT SEDGEWICK KEVIN WAYNE 2.2 MERGESORT Algorithms F O U R T H E D I T I O N mergesort bottom-up mergesort sorting complexity divide-and-conquer ROBERT SEDGEWICK KEVIN WAYNE http://algs4.cs.princeton.edu
More informationCOMP 382: Reasoning about algorithms
Fall 2014 Unit 4: Basics of complexity analysis Correctness and efficiency So far, we have talked about correctness and termination of algorithms What about efficiency? Running time of an algorithm For
More informationIntroduction to Algorithms 6.046J/18.401J/SMA5503
Introduction to Algorithms 6.046J/8.40J/SMA5503 Lecture 3 Prof. Piotr Indyk The divide-and-conquer design paradigm. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving
More informationDivide and conquer. Philip II of Macedon
Divide and conquer Philip II of Macedon Divide and conquer 1) Divide your problem into subproblems 2) Solve the subproblems recursively, that is, run the same algorithm on the subproblems (when the subproblems
More informationDivide & Conquer. CS 320, Fall Dr. Geri Georg, Instructor CS320 Div&Conq 1
Divide & Conquer CS 320, Fall 2017 Dr. Geri Georg, Instructor georg@colostate.edu CS320 Div&Conq 1 Strategy 1. Divide the problem up into equal sized sub problems 2. Solve the sub problems recursively
More information2. ALGORITHM ANALYSIS
2. ALGORITHM ANALYSIS computational tractability asymptotic order of growth survey of common running times Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationPartition and Select
Divide-Conquer-Glue Algorithms Quicksort, Quickselect and the Master Theorem Quickselect algorithm Tyler Moore CSE 3353, SMU, Dallas, TX Lecture 11 Selection Problem: find the kth smallest number of an
More informationReductions, Recursion and Divide and Conquer
Chapter 5 Reductions, Recursion and Divide and Conquer CS 473: Fundamental Algorithms, Fall 2011 September 13, 2011 5.1 Reductions and Recursion 5.1.0.1 Reduction Reducing problem A to problem B: (A) Algorithm
More informationInf 2B: Sorting, MergeSort and Divide-and-Conquer
Inf 2B: Sorting, MergeSort and Divide-and-Conquer Lecture 7 of ADS thread Kyriakos Kalorkoti School of Informatics University of Edinburgh The Sorting Problem Input: Task: Array A of items with comparable
More informationClass Note #14. In this class, we studied an algorithm for integer multiplication, which. 2 ) to θ(n
Class Note #14 Date: 03/01/2006 [Overall Information] In this class, we studied an algorithm for integer multiplication, which improved the running time from θ(n 2 ) to θ(n 1.59 ). We then used some of
More informationCopyright 2000, Kevin Wayne 1
Chapter 2 2.1 Computational Tractability Basics of Algorithm Analysis "For me, great algorithms are the poetry of computation. Just like verse, they can be terse, allusive, dense, and even mysterious.
More informationMA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3
MA008 p.1/37 MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3 Dr. Markus Hagenbuchner markus@uow.edu.au. MA008 p.2/37 Exercise 1 (from LN 2) Asymptotic Notation When constants appear in exponents
More informationCSC373: Algorithm Design, Analysis and Complexity Fall 2017
CSC373: Algorithm Design, Analysis and Complexity Fall 2017 Allan Borodin (in collaboration with Rabia Bakhteri and with occasional guest lectures by Nisarg Shah and Denis Pankratov) September 13, 2017
More informationDesign and Analysis of Algorithms Recurrence. Prof. Chuhua Xian School of Computer Science and Engineering
Design and Analysis of Algorithms Recurrence Prof. Chuhua Xian Email: chhxian@scut.edu.cn School of Computer Science and Engineering Course Information Instructor: Chuhua Xian ( 冼楚华 ) Email: chhxian@scut.edu.cn
More informationAlgorithms Design & Analysis. Analysis of Algorithm
Algorithms Design & Analysis Analysis of Algorithm Review Internship Stable Matching Algorithm 2 Outline Time complexity Computation model Asymptotic notions Recurrence Master theorem 3 The problem of
More informationGrade 11/12 Math Circles Fall Nov. 5 Recurrences, Part 2
1 Faculty of Mathematics Waterloo, Ontario Centre for Education in Mathematics and Computing Grade 11/12 Math Circles Fall 2014 - Nov. 5 Recurrences, Part 2 Running time of algorithms In computer science,
More informationCopyright 2000, Kevin Wayne 1
Algorithm runtime analysis and computational tractability Time Complexity of an Algorithm How do we measure the complexity (time, space requirements) of an algorithm. 1 microsecond? Units of time As soon
More informationTutorials. Algorithms and Data Structures Strassen s Algorithm. The Master Theorem for solving recurrences. The Master Theorem (cont d)
DS 2018/19 Lecture 4 slide 3 DS 2018/19 Lecture 4 slide 4 Tutorials lgorithms and Data Structures Strassen s lgorithm Start in week week 3 Tutorial allocations are linked from the course webpage http://www.inf.ed.ac.uk/teaching/courses/ads/
More informationCS 470/570 Divide-and-Conquer. Format of Divide-and-Conquer algorithms: Master Recurrence Theorem (simpler version)
CS 470/570 Divide-and-Conquer Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively on smaller pieces Combine: Build the
More informationData Structures and Algorithms
Data Structures and Algorithms Spring 2017-2018 Outline 1 Sorting Algorithms (contd.) Outline Sorting Algorithms (contd.) 1 Sorting Algorithms (contd.) Analysis of Quicksort Time to sort array of length
More informationMatching Residents to Hospitals
Midterm Review Matching Residents to Hospitals Goal. Given a set of preferences among hospitals and medical school students, design a self-reinforcing admissions process. Unstable pair: applicant x and
More informationCOMP Analysis of Algorithms & Data Structures
COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 10, 2018 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. 1 /
More information1 Substitution method
Recurrence Relations we have discussed asymptotic analysis of algorithms and various properties associated with asymptotic notation. As many algorithms are recursive in nature, it is natural to analyze
More informationCOMP Analysis of Algorithms & Data Structures
COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 14, 2019 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. COMP
More information10: Analysis of Algorithms
Overview 10: Analysis of Algorithms Which algorithm should I choose for a particular application? Ex: symbol table. (linked list, hash table, red-black tree,... ) Ex: sorting. (insertion sort, quicksort,
More informationWhat we have learned What is algorithm Why study algorithm The time and space efficiency of algorithm The analysis framework of time efficiency Asympt
Lecture 3 The Analysis of Recursive Algorithm Efficiency What we have learned What is algorithm Why study algorithm The time and space efficiency of algorithm The analysis framework of time efficiency
More informationDivide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,
Divide and Conquer CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Merging sorted lists: WHAT Given two sorted lists a 1 a 2 a 3 a k b 1 b 2 b 3 b
More informationThe Master Theorem for solving recurrences. Algorithms and Data Structures Strassen s Algorithm. Tutorials. The Master Theorem (cont d)
The Master Theorem for solving recurrences lgorithms and Data Structures Strassen s lgorithm 23rd September, 2014 Theorem Let n 0 N, k N 0 and a, b R with a > 0 and b > 1, and let T : N R satisfy the following
More informationCSE 421 Algorithms. T(n) = at(n/b) + n c. Closest Pair Problem. Divide and Conquer Algorithms. What you really need to know about recurrences
CSE 421 Algorithms Richard Anderson Lecture 13 Divide and Conquer What you really need to know about recurrences Work per level changes geometrically with the level Geometrically increasing (x > 1) The
More information/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17
601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17 2.1 Notes Homework 1 will be released today, and is due a week from today by the beginning
More informationDivide and Conquer Algorithms. CSE 101: Design and Analysis of Algorithms Lecture 14
Divide and Conquer Algorithms CSE 101: Design and Analysis of Algorithms Lecture 14 CSE 101: Design and analysis of algorithms Divide and conquer algorithms Reading: Sections 2.3 and 2.4 Homework 6 will
More informationLinear Selection and Linear Sorting
Analysis of Algorithms Linear Selection and Linear Sorting If more than one question appears correct, choose the more specific answer, unless otherwise instructed. Concept: linear selection 1. Suppose
More informationDivide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201
Divide and Conquer: Polynomial Multiplication Version of October 7, 2014 Divide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201 Outline Outline: Introduction The polynomial multiplication
More informationCS483 Design and Analysis of Algorithms
CS483 Design and Analysis of Algorithms Chapter 2 Divide and Conquer Algorithms Instructor: Fei Li lifei@cs.gmu.edu with subject: CS483 Office hours: Room 5326, Engineering Building, Thursday 4:30pm -
More informationCS 231: Algorithmic Problem Solving
CS 231: Algorithmic Problem Solving Naomi Nishimura Module 4 Date of this version: June 11, 2018 WARNING: Drafts of slides are made available prior to lecture for your convenience. After lecture, slides
More informationDivide and Conquer algorithms
Divide and Conquer algorithms Another general method for constructing algorithms is given by the Divide and Conquer strategy. We assume that we have a problem with input that can be split into parts in
More informationAsymptotic Algorithm Analysis & Sorting
Asymptotic Algorithm Analysis & Sorting (Version of 5th March 2010) (Based on original slides by John Hamer and Yves Deville) We can analyse an algorithm without needing to run it, and in so doing we can
More informationDivide and Conquer. Maximum/minimum. Median finding. CS125 Lecture 4 Fall 2016
CS125 Lecture 4 Fall 2016 Divide and Conquer We have seen one general paradigm for finding algorithms: the greedy approach. We now consider another general paradigm, known as divide and conquer. We have
More information