CS 470/570 Divide-and-Conquer. Format of Divide-and-Conquer algorithms: Master Recurrence Theorem (simpler version)
|
|
- Crystal Farmer
- 5 years ago
- Views:
Transcription
1 CS 470/570 Divide-and-Conquer Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively on smaller pieces Combine: Build the full solution from the recursive results Master Recurrence Theorem (simpler version) If T(n) = a T(n/b) + θ(n k ) then the solution is: T(n) = θ(n k ) if log b a < k (or a < b k ) T(n) = θ(n k lg n) if log b a = k (or a = b k ) T(n) = θ(n log b a ) if log b a > k (or a > b k ) Proof of Master Recurrence Theorem (simpler version): T(n) = a T(n/b) + cn k = a [a T(n/b 2 ) + c(n/b) k ] + cn k = a 2 T(n/b 2 ) + cn k (a/b k + 1)
2 = a 2 [a T(n/b 3 ) + c(n/b 2 ) k ] + cn k (1 + a/b k ) = a 3 T(n/b 3 ) + cn k (1 + a/b k + (a/b k ) 2 ) = = a t T(n/b t ) + cn k Σ 0 j t 1 (a/b k ) j [ Let n/b t = 1, so t = log b n, and a t = a log b n = n log b a. ] = n log b a T(1) + cn k Σ 0 j t 1 (a/b k ) j Consider S = Σ 0 j t 1 (a/b k ) j Three cases for sum of geometric series with ratio r = a/b k : If r < 1 (decreasing series) then a < b k, log b a < k, S = θ(first term) = θ((a/b k ) 0 ) = θ(1), and T(n) = θ(n log b a + n k 1) = θ(n k ). If r = 1 then a = b k, log b a = k S = θ(number of terms) = θ(t) = θ(log b n) = θ(lg n), and T(n) = θ(n log b a + n k lg n) = θ(n k lg n). If r > 1 (increasing series) then a > b k, log b a > k, S = θ(last term) = θ((a/b k ) t 1 ) = θ((a/b k ) t ) = θ((a b k ) log b n ) = θ(n log b a b k ) = θ(n log b a k ), and T(n) = θ(n log b a + n k n log b a k ) = θ(n log b a ).
3 Examples: Merge sort: T(n) = 2 T(n/2) + θ(n) a=2, b=2, k=1 log 2 2 = 1 So T(n) = θ(n k lg n) = θ(n lg n) Binary search: T(n) = T(n/2) + θ(1) a=1, b=2, k=0 log 2 1 = 0 So T(n) = θ(n k lg n) = θ(lg n)
4 But what if recurrence is not in form of master recurrence? Example: T(n) = a T(n 1/b ) + lg k n Change of variable, let n = 2 m and m = lg n T(2 m ) = a T(2 m/b ) + m k Now let S(m) = T(2 m ) = T(n) S(m) = a S(m/b) + m k By master recurrence theorem: S(m) = θ(m log b a ) if a > b k S(m) = θ(m k lg m) if a = b k S(m) = θ(m k ) if a < b k T(n) = θ((lg n) log b a ) T(n) = θ((lg n) k lg lg n) T(n) = θ((lg n) k ) if a > b k if a = b k if a < b k
5 Example: T(n) = a T(n d) + c n Change of variable, let n = lg m and m = 2 n T(lg m) = a T(lg m lg 2 d ) + c lg m = a T(lg (m/2 d )) + m lg c Let S(m) = T(lg m) = T(n) S(m) = a S(m/2 d ) + m lg c Let b = 2 d and d = lg b Let k = lg c and c = 2 k S(m) = a S(m/b) + m k By master recurrence theorem: S(m) = θ(m k ) if a < b k S(m) = θ(m k lg m) if a = b k S(m) = θ(m log b a ) if a > b k Note b k = 2 dk = c d Also m k = 2 nk = c n Also m log b a = a log b m = a lg m/lg b = a n/d Therefore: T(n) = θ(c n ) T(n) = θ(n c n ) T(n) = θ(a n/d ) if a < c d if a = c d if a > c d
6 Divide-and-conquer algorithms Polynomial multiplication problem Given two polynomials: P = 5x 3 + 7x 2 + 6x + 2 [5, 7, 6, 2] = [p 3, p 2, p 1, p 0 ] Q = x 3 8x 2 + 9x 1 [1, 8, 9, 1] = [q 3, q 2, q 1, q 0 ] P*Q = (5x 3 + 7x 2 + 6x + 2)(x 3 8x 2 + 9x 1) Divide-and-conquer Algorithm 1 for Polynomial multiplication: P = 5x 3 + 7x 2 + 6x + 2 = (5x + 7)x 2 + (6x + 2) = Ax 2 + B Q = x 3 8x 2 + 9x 1 = (x 8)x 2 + (9x 1) = Cx 2 + D A = 5x + 7 [5, 7] B = 6x + 2 [6, 2] C = x 8 [1, 8] D = 9x 1 [9, 1] Let n = number of terms in P and Q P = Ax n/2 + B Q = Cx n/2 + D A, B, C, D are polynomials with n/2 terms P*Q = (Ax n/2 + B)(Cx n/2 + D) = (AC)x n + (BC + AD)x n/2 + (BD)
7 Four recursive subproblems: A*C B*C A*D B*D Stop the recursion when n = 1 Analysis of Algorithm 1 for polynomial multiplication: T(n) = 4 T(n/2) + θ(n) a=4, b=2, k=1 log 2 4 = 2 > 1 So T(n) = θ(n log b a ) = θ(n 2 )
8 Divide-and-conquer Algorithm 2 (Karatsuba s algorithm) for Polynomial multiplication: As before, P*Q = (Ax n/2 + B)(Cx n/2 + D) = (AC)x n + (BC + AD)x n/2 + (BD) This time we write (BC + AD) = (A+B)(C+D) (AC) (BD) So only three recursive calls: A*C B*D (A+B)*(C+D) Again stop the recursion when n = 1 Analysis of Algorithm 2 for polynomial multiplication: T(n) = 3 T(n/2) + θ(n) a=3, b=2, k=1 log 2 3 > 1 So T(n) = θ(n log b a ) = θ(n lg 3 ) θ(n 1.58 )
9 Matrix multiplication problem If matrices A, B are square n-by-n matrices, then product C = A*B is also square n-by-n matrix Example with n=3: = Each element C[i][j] = Σ 1 k n A[i][k]*B[k][j] The above formula leads to the following naive algorithm for computing the C matrix in θ(n 3 ) total time: for i = 1 to n for j = 1 to n { C[i][j] = 0; for k = 1 to n C[i][j] += A[i][k]*B[k][j]; }
10 Divide-and-conquer Algorithm 1 for matrix multiplication: Assume n is even Otherwise, add extra row and extra column with all 0s Stop the recursion when n=1 aa 1,1 aa 1,nn bb 1,1 bb 1,nn If A = and B = aa nn,1 aa nn,nn bb nn,1 bb nn,nn then write A = SS TT XX and B = WW where each of S, T, U, UU VV YY ZZ V, W, X, Y, Z is a square (n/2)-by-(n/2) matrix aa 1,1 aa 1,nn/2 For example, S = aa nn/2,1 aa nn/2,nn/2 C = A*B = SS TT UU VV Eight recursive calls: S*W T*Y S*X T*Z U*W V*Y U*X V*Z WW XX YY ZZ WW + TT YY SS XX + TT ZZ = SS UU WW + VV YY UU XX + VV ZZ
11 Analysis of Algorithm 1 for matrix multiplication: T(n) = 8 T(n/2) + θ(n 2 ) a=8, b=2, k=2 log 2 8 = 3 > 2 So T(n) = θ(n 3 ) Same running time as the previous naïve algorithm
12 Divide-and-conquer Algorithm 2 (Strassen s algorithm) for matrix multiplication: Again we write A = SS TT XX and B = WW UU VV YY ZZ 7 recursive calls: P1 = S * (X Z) P2 = (S + T) * Z P3 = (U + V) * W P4 = V * (Y W) P5 = (S + V) * (W + Z) P6 = (T V) * (Y + Z) P7 = (U S) * (W + X) PP5 + PP4 PP2 + PP6 C = PP3 + PP4 PP1 + PP2 PP5 + PP1 PP3 + PP7 Difficult to derive, but easy to verify the algebra: P1 + P2 = S*X + T*Z P3 + P4 = U*W + V*Y etc.
13 Analysis of Algorithm 2 for matrix multiplication: T(n) = 7 T(n/2) + θ(n 2 ) a=7, b=2, k=2 log 2 7 > 2 So T(n) = θ(n lg 7 ) θ(n 2.81 ) Current best known algorithm for matrix multiplication θ(n 2.37 ) time, by further extending this D&C approach, but the formulas are too complicated to present here
14 Majority element problem: Given array A of size n, does there exist any value M that appears more than n/2 times in array A? Majority element algorithm: Phase 1: Use divide-and-conquer to find candidate value M Phase 2: Check if M really is a majority element, θ(n) time, simple loop Phase 1 details: Divide: o Group the elements of A into n/2 pairs o If n is odd, there is one unpaired element, x Check if this x is majority element of A If so, then return x, but otherwise discard x o Compare each pair (y, z) If (y==z) keep y and discard z If (y!=z) discard both y and z o So we keep n/2 elements Conquer: One recursive call on subarray of size n/2 Combine: Nothing remains to be done, so omit this step
15 Example: A = [7, 7, 5, 2, 5, 5, 4, 5, 5, 5, 7] (7, 7) (5, 2) (5, 5) (4, 5) (5, 5) (7) A = [7, 5, 5] (7, 5) (5) return 5 (candidate, also majority) Example: A = [1, 2, 3, 1, 2, 3, 1, 2, 9, 9] (1, 2) (3, 1) (2, 3) (1, 2) (9, 9) A = [9] return 9 (candidate, but not majority) Analysis: Let T(n) = running time of Phase 1 on array of size n T(n) = T(n/2) + θ(n) Number of recursive subproblems = 1 Size of each subproblem = n/2 [worst-case] Time for all the non-recursive steps = θ(n) Solution: T(n) = T(n/2) + θ(n) a=1, b=2, k=1 log 2 1 = 0 < 1 So T(n) = θ(n k ) = θ(n)
16 Selection problem: Given array A[1 n] and value k where 1 k n, find and return the k th smallest element in A If k=1 minimum element If k=n maximum element If k=(1+n)/2 median element
17 Algorithm 1 for selection problem: sort A into ascending order; return A[k]; Algorithm 1 takes θ(n lg n) time using merge sort or heap sort Can we develop a faster algorithm? Note: without sorting, we can determine the minimum or maximum element in θ(n) time. [How?] Goal: solve the selection problem for arbitrary k in θ(n) time Idea: divide-and-conquer, use pivot similar to quick sort, but only make recursive call on one of the two subarrays, because we don t need to completely sort the entire array
18 Algorithm 2 for selection problem: Choose pivot element that is hopefully near the median of A Recall these strategies for choosing the pivot in quick sort: pivot = A[low] pivot = A[high] pivot = A[(low+high)/2] pivot = A[random (low, high)] pivot = median (A[random (low, high)], A[random (low, high)], A[random (low, high)]) pivot = median (A[low], A[(low+high)/2], A[high]) Select (A, k) { pivot = ; // choose any of the above strategies create three empty lists: L, E, G; for each x in A if (x<pivot) add x to L; else if (x==pivot) add x to E; else /* (x>pivot) */ add x to G; if (k <= L.size) return Select (L, k); else if (k <= L.size + E.size) return pivot; else return Select (G, k L.size E.size); }
19 Analysis of Algorithm 2 for selection problem: Best case: θ(n), if pivot happens to be the k th smallest element Worst case: θ(n 2 ), if pivot is always near the minimum or maximum value Average case: θ(n), if sometimes pivot yields a good split and sometimes a bad split, based on probabilities Recurrence for worst case: T(n) = T(n 1) + θ(n) or T(n) = T(n 2) + θ(n) Recurrence for average case (assuming no duplicates): T(n) = 1 Σ 1 k n [ k 1 T(k 1) + n k T(n k)] + θ(n) n n n Does not conform to the master recurrence theorem, so it s difficult to solve How can we achieve worst-case θ(n) time for selection? Note: if we could be very lucky to always guess the median element as the pivot, then T(n) = T(n/2) + θ(n) T(n) = θ(n) So we want a new strategy for choosing a pivot that s always close to the median
20 Algorithm 3 for selection problem: Same as algorithm 2, except for new pivot strategy: Choose an odd number g (later we ll see g=5 is best) Partition the n elements into groups of size g each (So the number of groups = n/g) Find the median of each group (Note: we can sort each group in θ(g 2 ) = θ(1) time) Let M = list of all these group medians, so size of M is n/g Find the median of M by calling Algorithm 3 recursively (Note: because we can t sort M in θ(n) time) Let pivot = the median of M = Select (M, (1 + n/g)/2) (So pivot is the median-of-medians) Next continue the same as in Algorithm 2: create three empty lists: L, E, G; for each x in A if (x<pivot) add x to L; else if (x==pivot) add x to E; else /* (x>pivot) */ add x to G; if (k <= L.size) return Select (L, k); else if (k <= L.size + E.size) return pivot; else return Select (G, k L.size E.size);
21 Stop the recursion when n is below some threshold (such as n < 3g or n < g 2 ), and solve using Algorithm 1 or Algorithm 2 Example: n=25, let g=5 A To find the median of A, call Select (A, (1+25)/2) = Select (A, 13) A M = [13, 17, 3, 16, 18] pivot = Select (M, (1+25/5)/2) = Select ([13,17,3,16,18], 3) = 16 L = [1,14,11,15,13,4,6,0,10,8,3,2,9,12,5,7] L.size = 16 E = [16] E.size = 1 G = [23,17,19,21,22,24,18,20] G.size = 8 k=13 k <= L.size Select (L, 13) = 12 Next suppose we call Select (A, 21) using same array A Almost everything proceeds exactly as above k=21 k > L.size + E.size Select (G, ) Select (G, 4) = 20
22 Analysis of Algorithm 3 for selection problem: Two recursive calls pivot = Select (M, (1 + n/g)/2) only one of Select (L, k) or Select (G, k L.size E.size) T(n) = T(M.size) + T(max(L.size, G.size)) + θ(n) Recall M.size = n/g What is upper bound for L.size and G.size? Note: pivot is the median of M So half of the n/g elements in M must be pivot Half of the n/g groups have medians pivot Each of these groups has at least g/2 elements pivot Altogether, at least (1/2)(n/g)(g/2) = n/4 elements pivot All these n/4 elements are in L and E (so they re not in G) Therefore G.size 3n/4 Analogously we can show that L.size 3n/4 so max(l.size, G.size) 3n/4
23 Intuition: Select (A, n/4) pivot Select (A, 3n/4), so pivot is closer to median than it is to min or max elements T(n) = T(n/g) + T(3n/4) + θ(n) Does not conform to the master recurrence theorem, but we can solve it easily by another approach T(n) = T(n/g) + T(3n/4) + cn, for some constant c > 0 Guess that T(n) = dn, for some other constant d > 0 dn = d(n/g) + d(3n/4) + cn d = d/g + 3d/4 + c d ( 1/4 1/g ) = c Note: must have g > 4 for this equation to be solvable, so choose group size g=5 d ( 1/4 1/5 ) = c d ( 1/20 ) = c d = 20c So T(n) = dn = 20cn = θ(n) Algorithm 3 is a worst-case θ(n)-time algorithm
Data Structures and Algorithms CSE 465
Data Structures and Algorithms CSE 465 LECTURE 8 Analyzing Quick Sort Sofya Raskhodnikova and Adam Smith Reminder: QuickSort Quicksort an n-element array: 1. Divide: Partition the array around a pivot
More informationDivide-and-conquer: Order Statistics. Curs: Fall 2017
Divide-and-conquer: Order Statistics Curs: Fall 2017 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers.
More informationDivide and Conquer. Arash Rafiey. 27 October, 2016
27 October, 2016 Divide the problem into a number of subproblems Divide the problem into a number of subproblems Conquer the subproblems by solving them recursively or if they are small, there must be
More informationA design paradigm. Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/ EECS 3101
A design paradigm Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/17 112 Multiplying complex numbers (from Jeff Edmonds slides) INPUT: Two pairs of integers, (a,b),
More informationIntroduction to Algorithms 6.046J/18.401J/SMA5503
Introduction to Algorithms 6.046J/8.40J/SMA5503 Lecture 3 Prof. Piotr Indyk The divide-and-conquer design paradigm. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving
More informationAlgorithms And Programming I. Lecture 5 Quicksort
Algorithms And Programming I Lecture 5 Quicksort Quick Sort Partition set into two using randomly chosen pivot 88 31 25 52 14 98 62 30 23 79 14 31 2530 23 52 88 62 98 79 Quick Sort 14 31 2530 23 52 88
More informationChapter 4 Divide-and-Conquer
Chapter 4 Divide-and-Conquer 1 About this lecture (1) Recall the divide-and-conquer paradigm, which we used for merge sort: Divide the problem into a number of subproblems that are smaller instances of
More informationDivide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch] Divide and Conquer Paradigm An important general technique for designing algorithms: divide problem into subproblems recursively
More informationThe maximum-subarray problem. Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm:
The maximum-subarray problem Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm: Brute force algorithm: At best, θ(n 2 ) time complexity 129 Can we do divide
More informationCS 577 Introduction to Algorithms: Strassen s Algorithm and the Master Theorem
CS 577 Introduction to Algorithms: Jin-Yi Cai University of Wisconsin Madison In the last class, we described InsertionSort and showed that its worst-case running time is Θ(n 2 ). Check Figure 2.2 for
More informationCMPS 2200 Fall Divide-and-Conquer. Carola Wenk. Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk
CMPS 2200 Fall 2017 Divide-and-Conquer Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk 1 The divide-and-conquer design paradigm 1. Divide the problem (instance)
More informationChapter 2. Recurrence Relations. Divide and Conquer. Divide and Conquer Strategy. Another Example: Merge Sort. Merge Sort Example. Merge Sort Example
Recurrence Relations Chapter 2 Divide and Conquer Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative
More informationDivide and conquer. Philip II of Macedon
Divide and conquer Philip II of Macedon Divide and conquer 1) Divide your problem into subproblems 2) Solve the subproblems recursively, that is, run the same algorithm on the subproblems (when the subproblems
More informationb + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d
CS161, Lecture 4 Median, Selection, and the Substitution Method Scribe: Albert Chen and Juliana Cook (2015), Sam Kim (2016), Gregory Valiant (2017) Date: January 23, 2017 1 Introduction Last lecture, we
More informationDivide and Conquer. Andreas Klappenecker
Divide and Conquer Andreas Klappenecker The Divide and Conquer Paradigm The divide and conquer paradigm is important general technique for designing algorithms. In general, it follows the steps: - divide
More informationDivide and Conquer algorithms
Divide and Conquer algorithms Another general method for constructing algorithms is given by the Divide and Conquer strategy. We assume that we have a problem with input that can be split into parts in
More informationDivide-and-conquer. Curs 2015
Divide-and-conquer Curs 2015 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers. Known Examples: Binary
More informationReview Of Topics. Review: Induction
Review Of Topics Asymptotic notation Solving recurrences Sorting algorithms Insertion sort Merge sort Heap sort Quick sort Counting sort Radix sort Medians/order statistics Randomized algorithm Worst-case
More information1 Divide and Conquer (September 3)
The control of a large force is the same principle as the control of a few men: it is merely a question of dividing up their numbers. Sun Zi, The Art of War (c. 400 C.E.), translated by Lionel Giles (1910)
More informationLecture 4. Quicksort
Lecture 4. Quicksort T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018 Networking
More informationCS 161 Summer 2009 Homework #2 Sample Solutions
CS 161 Summer 2009 Homework #2 Sample Solutions Regrade Policy: If you believe an error has been made in the grading of your homework, you may resubmit it for a regrade. If the error consists of more than
More informationCSCI 3110 Assignment 6 Solutions
CSCI 3110 Assignment 6 Solutions December 5, 2012 2.4 (4 pts) Suppose you are choosing between the following three algorithms: 1. Algorithm A solves problems by dividing them into five subproblems of half
More informationData structures Exercise 1 solution. Question 1. Let s start by writing all the functions in big O notation:
Data structures Exercise 1 solution Question 1 Let s start by writing all the functions in big O notation: f 1 (n) = 2017 = O(1), f 2 (n) = 2 log 2 n = O(n 2 ), f 3 (n) = 2 n = O(2 n ), f 4 (n) = 1 = O
More informationDivide-and-Conquer. a technique for designing algorithms
Divide-and-Conquer a technique for designing algorithms decomposing instance to be solved into subinstances of the same problem solving each subinstance combining subsolutions to obtain the solution to
More informationDivide & Conquer. Jordi Cortadella and Jordi Petit Department of Computer Science
Divide & Conquer Jordi Cortadella and Jordi Petit Department of Computer Science Divide-and-conquer algorithms Strategy: Divide the problem into smaller subproblems of the same type of problem Solve the
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 4: Divide and Conquer (I) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ Divide and Conquer ( DQ ) First paradigm or framework DQ(S)
More informationAlgorithm Design and Analysis
Algorithm Design and Analysis LECTURE 9 Divide and Conquer Merge sort Counting Inversions Binary Search Exponentiation Solving Recurrences Recursion Tree Method Master Theorem Sofya Raskhodnikova S. Raskhodnikova;
More informationQuick Sort Notes , Spring 2010
Quick Sort Notes 18.310, Spring 2010 0.1 Randomized Median Finding In a previous lecture, we discussed the problem of finding the median of a list of m elements, or more generally the element of rank m.
More informationDivide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201
Divide and Conquer: Polynomial Multiplication Version of October 7, 2014 Divide and Conquer: Polynomial Multiplication Version of October 1 / 7, 24201 Outline Outline: Introduction The polynomial multiplication
More informationDivide and Conquer Algorithms. CSE 101: Design and Analysis of Algorithms Lecture 14
Divide and Conquer Algorithms CSE 101: Design and Analysis of Algorithms Lecture 14 CSE 101: Design and analysis of algorithms Divide and conquer algorithms Reading: Sections 2.3 and 2.4 Homework 6 will
More informationAlgorithms Test 1. Question 1. (10 points) for (i = 1; i <= n; i++) { j = 1; while (j < n) {
Question 1. (10 points) for (i = 1; i
More informationChapter 1 Divide and Conquer Algorithm Theory WS 2016/17 Fabian Kuhn
Chapter 1 Divide and Conquer Algorithm Theory WS 2016/17 Fabian Kuhn Formulation of the D&C principle Divide-and-conquer method for solving a problem instance of size n: 1. Divide n c: Solve the problem
More informationAnalysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort
Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Xi Chen Columbia University We continue with two more asymptotic notation: o( ) and ω( ). Let f (n) and g(n) are functions that map
More informationCS161: Algorithm Design and Analysis Recitation Section 3 Stanford University Week of 29 January, Problem 3-1.
CS161: Algorithm Design and Analysis Recitation Section 3 Stanford University Week of 29 January, 2018 Problem 3-1. (Quicksort Median-of-3 Partition) One way to improve the randomized quicksort procedure
More informationAsymptotic Analysis and Recurrences
Appendix A Asymptotic Analysis and Recurrences A.1 Overview We discuss the notion of asymptotic analysis and introduce O, Ω, Θ, and o notation. We then turn to the topic of recurrences, discussing several
More informationAnalysis of Algorithms - Using Asymptotic Bounds -
Analysis of Algorithms - Using Asymptotic Bounds - Andreas Ermedahl MRTC (Mälardalens Real-Time Research Center) andreas.ermedahl@mdh.se Autumn 004 Rehersal: Asymptotic bounds Gives running time bounds
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 5: Divide and Conquer (Part 2) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ A Lower Bound on Convex Hull Lecture 4 Task: sort the
More informationClass Note #14. In this class, we studied an algorithm for integer multiplication, which. 2 ) to θ(n
Class Note #14 Date: 03/01/2006 [Overall Information] In this class, we studied an algorithm for integer multiplication, which improved the running time from θ(n 2 ) to θ(n 1.59 ). We then used some of
More informationAlgorithms. Quicksort. Slide credit: David Luebke (Virginia)
1 Algorithms Quicksort Slide credit: David Luebke (Virginia) Sorting revisited We have seen algorithms for sorting: INSERTION-SORT, MERGESORT More generally: given a sequence of items Each item has a characteristic
More informationQuicksort (CLRS 7) We previously saw how the divide-and-conquer technique can be used to design sorting algorithm Merge-sort
Quicksort (CLRS 7) We previously saw how the divide-and-conquer technique can be used to design sorting algorithm Merge-sort Partition n elements array A into two subarrays of n/2 elements each Sort the
More informationCS 5321: Advanced Algorithms - Recurrence. Acknowledgement. Outline. Ali Ebnenasir Department of Computer Science Michigan Technological University
CS 5321: Advanced Algorithms - Recurrence Ali Ebnenasir Department of Computer Science Michigan Technological University Acknowledgement Eric Torng Moon Jung Chung Charles Ofria Outline Motivating example:
More informationDivide and Conquer. Maximum/minimum. Median finding. CS125 Lecture 4 Fall 2016
CS125 Lecture 4 Fall 2016 Divide and Conquer We have seen one general paradigm for finding algorithms: the greedy approach. We now consider another general paradigm, known as divide and conquer. We have
More informationDivide & Conquer. Jordi Cortadella and Jordi Petit Department of Computer Science
Divide & Conquer Jordi Cortadella and Jordi Petit Department of Computer Science Divide-and-conquer algorithms Strategy: Divide the problem into smaller subproblems of the same type of problem Solve the
More informationThe Divide-and-Conquer Design Paradigm
CS473- Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm CS473 Lecture 4 1 The Divide-and-Conquer Design Paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems
More informationDivide and Conquer Strategy
Divide and Conquer Strategy Algorithm design is more an art, less so a science. There are a few useful strategies, but no guarantee to succeed. We will discuss: Divide and Conquer, Greedy, Dynamic Programming.
More informationCS 5321: Advanced Algorithms Analysis Using Recurrence. Acknowledgement. Outline
CS 5321: Advanced Algorithms Analysis Using Recurrence Ali Ebnenasir Department of Computer Science Michigan Technological University Acknowledgement Eric Torng Moon Jung Chung Charles Ofria Outline Motivating
More informationCPS 616 DIVIDE-AND-CONQUER 6-1
CPS 616 DIVIDE-AND-CONQUER 6-1 DIVIDE-AND-CONQUER Approach 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger)
More informationCS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms
CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions
More informationData Structures and Algorithms Chapter 3
Data Structures and Algorithms Chapter 3 1. Divide and conquer 2. Merge sort, repeated substitutions 3. Tiling 4. Recurrences Recurrences Running times of algorithms with recursive calls can be described
More informationAnalysis of Multithreaded Algorithms
Analysis of Multithreaded Algorithms Marc Moreno Maza University of Western Ontario, London, Ontario (Canada) CS 4435 - CS 9624 (Moreno Maza) Analysis of Multithreaded Algorithms CS 4435 - CS 9624 1 /
More informationAnalysis of Algorithms CMPSC 565
Analysis of Algorithms CMPSC 565 LECTURES 38-39 Randomized Algorithms II Quickselect Quicksort Running time Adam Smith L1.1 Types of randomized analysis Average-case analysis : Assume data is distributed
More informationAsymptotic Algorithm Analysis & Sorting
Asymptotic Algorithm Analysis & Sorting (Version of 5th March 2010) (Based on original slides by John Hamer and Yves Deville) We can analyse an algorithm without needing to run it, and in so doing we can
More informationCS483 Design and Analysis of Algorithms
CS483 Design and Analysis of Algorithms Lecture 6-8 Divide and Conquer Algorithms Instructor: Fei Li lifei@cs.gmu.edu with subject: CS483 Office hours: STII, Room 443, Friday 4:00pm - 6:00pm or by appointments
More informationCSE 613: Parallel Programming. Lectures ( Analyzing Divide-and-Conquer Algorithms )
CSE 613: Parallel Programming Lectures 13 14 ( Analyzing Divide-and-Conquer Algorithms ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2015 A Useful Recurrence Consider the
More informationCS 4104 Data and Algorithm Analysis. Recurrence Relations. Modeling Recursive Function Cost. Solving Recurrences. Clifford A. Shaffer.
Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2010,2017 by Clifford A. Shaffer Data and Algorithm Analysis Title page Data and Algorithm Analysis Clifford A. Shaffer Spring
More informationChapter 5. Divide and Conquer CLRS 4.3. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer CLRS 4.3 Slides by Kevin Wayne. Copyright 25 Pearson-Addison Wesley. All rights reserved. Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve
More information/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17
601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17 2.1 Notes Homework 1 will be released today, and is due a week from today by the beginning
More informationCS361 Homework #3 Solutions
CS6 Homework # Solutions. Suppose I have a hash table with 5 locations. I would like to know how many items I can store in it before it becomes fairly likely that I have a collision, i.e., that two items
More informationDivide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,
Divide and Conquer CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Merging sorted lists: WHAT Given two sorted lists a 1 a 2 a 3 a k b 1 b 2 b 3 b
More informationMergesort and Recurrences (CLRS 2.3, 4.4)
Mergesort and Recurrences (CLRS 2.3, 4.4) We saw a couple of O(n 2 ) algorithms for sorting. Today we ll see a different approach that runs in O(n lg n) and uses one of the most powerful techniques for
More informationCS 4407 Algorithms Lecture 3: Iterative and Divide and Conquer Algorithms
CS 4407 Algorithms Lecture 3: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions
More information1 Quick Sort LECTURE 7. OHSU/OGI (Winter 2009) ANALYSIS AND DESIGN OF ALGORITHMS
OHSU/OGI (Winter 2009) CS532 ANALYSIS AND DESIGN OF ALGORITHMS LECTURE 7 1 Quick Sort QuickSort 1 is a classic example of divide and conquer. The hard work is to rearrange the elements of the array A[1..n]
More informationDivide-and-Conquer Algorithms and Recurrence Relations. Niloufar Shafiei
Divide-and-Conquer Algorithms and Recurrence Relations Niloufar Shafiei Divide-and-conquer algorithms Divide-and-conquer algorithms: 1. Dividing the problem into smaller sub-problems 2. Solving those sub-problems
More informationIntroduction to Divide and Conquer
Introduction to Divide and Conquer Sorting with O(n log n) comparisons and integer multiplication faster than O(n 2 ) Periklis A. Papakonstantinou York University Consider a problem that admits a straightforward
More informationChapter 1 Divide and Conquer Polynomial Multiplication Algorithm Theory WS 2015/16 Fabian Kuhn
Chapter 1 Divide and Conquer Polynomial Multiplication Algorithm Theory WS 2015/16 Fabian Kuhn Formulation of the D&C principle Divide-and-conquer method for solving a problem instance of size n: 1. Divide
More informationFast Convolution; Strassen s Method
Fast Convolution; Strassen s Method 1 Fast Convolution reduction to subquadratic time polynomial evaluation at complex roots of unity interpolation via evaluation at complex roots of unity 2 The Master
More informationAlgorithms Chapter 4 Recurrences
Algorithms Chapter 4 Recurrences Instructor: Ching Chi Lin 林清池助理教授 chingchi.lin@gmail.com Department of Computer Science and Engineering National Taiwan Ocean University Outline The substitution method
More informationDivide-and-conquer algorithm
Divide-and-conquer algorithm IDEA: n n matrix = 2 2 matrix of (n/2) (n/2) submatrices: r=ae+bg s=af+bh t =ce+dh u=cf+dg r t s u = a c e g September 15, 2004 Introduction to Algorithms L3.31 b d C = A B
More informationCSE 421 Algorithms. T(n) = at(n/b) + n c. Closest Pair Problem. Divide and Conquer Algorithms. What you really need to know about recurrences
CSE 421 Algorithms Richard Anderson Lecture 13 Divide and Conquer What you really need to know about recurrences Work per level changes geometrically with the level Geometrically increasing (x > 1) The
More informationMA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3
MA008 p.1/37 MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3 Dr. Markus Hagenbuchner markus@uow.edu.au. MA008 p.2/37 Exercise 1 (from LN 2) Asymptotic Notation When constants appear in exponents
More informationReductions, Recursion and Divide and Conquer
Chapter 5 Reductions, Recursion and Divide and Conquer CS 473: Fundamental Algorithms, Fall 2011 September 13, 2011 5.1 Reductions and Recursion 5.1.0.1 Reduction Reducing problem A to problem B: (A) Algorithm
More informationSearching. Sorting. Lambdas
.. s Babes-Bolyai University arthur@cs.ubbcluj.ro Overview 1 2 3 Feedback for the course You can write feedback at academicinfo.ubbcluj.ro It is both important as well as anonymous Write both what you
More informationChapter 5. Divide and Conquer CLRS 4.3. Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved.
Chapter 5 Divide and Conquer CLRS 4.3 Slides by Kevin Wayne. Copyright 2005 Pearson-Addison Wesley. All rights reserved. 1 Divide-and-Conquer Divide-and-conquer. Break up problem into several parts. Solve
More informationV. Adamchik 1. Recurrences. Victor Adamchik Fall of 2005
V. Adamchi Recurrences Victor Adamchi Fall of 00 Plan Multiple roots. More on multiple roots. Inhomogeneous equations 3. Divide-and-conquer recurrences In the previous lecture we have showed that if the
More informationCOMP Analysis of Algorithms & Data Structures
COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 10, 2018 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. 1 /
More information5. DIVIDE AND CONQUER I
5. DIVIDE AND CONQUER I mergesort counting inversions closest pair of points median and selection Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationWeek 5: Quicksort, Lower bound, Greedy
Week 5: Quicksort, Lower bound, Greedy Agenda: Quicksort: Average case Lower bound for sorting Greedy method 1 Week 5: Quicksort Recall Quicksort: The ideas: Pick one key Compare to others: partition into
More informationCOMP Analysis of Algorithms & Data Structures
COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 14, 2019 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. COMP
More informationSolving recurrences. Frequently showing up when analysing divide&conquer algorithms or, more generally, recursive algorithms.
Solving recurrences Frequently showing up when analysing divide&conquer algorithms or, more generally, recursive algorithms Example: Merge-Sort(A, p, r) 1: if p < r then 2: q (p + r)/2 3: Merge-Sort(A,
More informationPartition and Select
Divide-Conquer-Glue Algorithms Quicksort, Quickselect and the Master Theorem Quickselect algorithm Tyler Moore CSE 3353, SMU, Dallas, TX Lecture 11 Selection Problem: find the kth smallest number of an
More informationCS483 Design and Analysis of Algorithms
CS483 Design and Analysis of Algorithms Chapter 2 Divide and Conquer Algorithms Instructor: Fei Li lifei@cs.gmu.edu with subject: CS483 Office hours: Room 5326, Engineering Building, Thursday 4:30pm -
More informationInf 2B: Sorting, MergeSort and Divide-and-Conquer
Inf 2B: Sorting, MergeSort and Divide-and-Conquer Lecture 7 of ADS thread Kyriakos Kalorkoti School of Informatics University of Edinburgh The Sorting Problem Input: Task: Array A of items with comparable
More informationAlgorithms, Design and Analysis. Order of growth. Table 2.1. Big-oh. Asymptotic growth rate. Types of formulas for basic operation count
Types of formulas for basic operation count Exact formula e.g., C(n) = n(n-1)/2 Algorithms, Design and Analysis Big-Oh analysis, Brute Force, Divide and conquer intro Formula indicating order of growth
More informationData Structures and Algorithms CSE 465
Data Structures and Algorithms CSE 465 LECTURE 3 Asymptotic Notation O-, Ω-, Θ-, o-, ω-notation Divide and Conquer Merge Sort Binary Search Sofya Raskhodnikova and Adam Smith /5/0 Review Questions If input
More informationCMPSCI611: Three Divide-and-Conquer Examples Lecture 2
CMPSCI611: Three Divide-and-Conquer Examples Lecture 2 Last lecture we presented and analyzed Mergesort, a simple divide-and-conquer algorithm. We then stated and proved the Master Theorem, which gives
More information5. DIVIDE AND CONQUER I
5. DIVIDE AND CONQUER I mergesort counting inversions closest pair of points randomized quicksort median and selection Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos
More informationCMPT 307 : Divide-and-Conqer (Study Guide) Should be read in conjunction with the text June 2, 2015
CMPT 307 : Divide-and-Conqer (Study Guide) Should be read in conjunction with the text June 2, 2015 1 Introduction The divide-and-conquer strategy is a general paradigm for algorithm design. This strategy
More informationChapter 5 Divide and Conquer
CMPT 705: Design and Analysis of Algorithms Spring 008 Chapter 5 Divide and Conquer Lecturer: Binay Bhattacharya Scribe: Chris Nell 5.1 Introduction Given a problem P with input size n, P (n), we define
More informationdata structures and algorithms lecture 2
data structures and algorithms 2018 09 06 lecture 2 recall: insertion sort Algorithm insertionsort(a, n): for j := 2 to n do key := A[j] i := j 1 while i 1 and A[i] > key do A[i + 1] := A[i] i := i 1 A[i
More informationThe divide-and-conquer strategy solves a problem by: 1. Breaking it into subproblems that are themselves smaller instances of the same type of problem
Chapter 2. Divide-and-conquer algorithms The divide-and-conquer strategy solves a problem by: 1. Breaking it into subproblems that are themselves smaller instances of the same type of problem. 2. Recursively
More informationComputational Complexity. This lecture. Notes. Lecture 02 - Basic Complexity Analysis. Tom Kelsey & Susmit Sarkar. Notes
Computational Complexity Lecture 02 - Basic Complexity Analysis Tom Kelsey & Susmit Sarkar School of Computer Science University of St Andrews http://www.cs.st-andrews.ac.uk/~tom/ twk@st-andrews.ac.uk
More informationBucket-Sort. Have seen lower bound of Ω(nlog n) for comparisonbased. Some cheating algorithms achieve O(n), given certain assumptions re input
Bucket-Sort Have seen lower bound of Ω(nlog n) for comparisonbased sorting algs Some cheating algorithms achieve O(n), given certain assumptions re input One example: bucket sort Assumption: input numbers
More informationDivide-and-Conquer Algorithms Part Two
Divide-and-Conquer Algorithms Part Two Recap from Last Time Divide-and-Conquer Algorithms A divide-and-conquer algorithm is one that works as follows: (Divide) Split the input apart into multiple smaller
More information1 Substitution method
Recurrence Relations we have discussed asymptotic analysis of algorithms and various properties associated with asymptotic notation. As many algorithms are recursive in nature, it is natural to analyze
More informationCSCI Honor seminar in algorithms Homework 2 Solution
CSCI 493.55 Honor seminar in algorithms Homework 2 Solution Saad Mneimneh Visiting Professor Hunter College of CUNY Problem 1: Rabin-Karp string matching Consider a binary string s of length n and another
More informationData Structures and Algorithms Chapter 3
1 Data Structures and Algorithms Chapter 3 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples
More informationRandomized Sorting Algorithms Quick sort can be converted to a randomized algorithm by picking the pivot element randomly. In this case we can show th
CSE 3500 Algorithms and Complexity Fall 2016 Lecture 10: September 29, 2016 Quick sort: Average Run Time In the last lecture we started analyzing the expected run time of quick sort. Let X = k 1, k 2,...,
More informationCOMP 250: Quicksort. Carlos G. Oliver. February 13, Carlos G. Oliver COMP 250: Quicksort February 13, / 21
COMP 250: Quicksort Carlos G. Oliver February 13, 2018 Carlos G. Oliver COMP 250: Quicksort February 13, 2018 1 / 21 1 1 https://xkcd.com/1667/ Carlos G. Oliver COMP 250: Quicksort February 13, 2018 2
More informationCSE 613: Parallel Programming. Lecture 8 ( Analyzing Divide-and-Conquer Algorithms )
CSE 613: Parallel Programming Lecture 8 ( Analyzing Divide-and-Conquer Algorithms ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2012 A Useful Recurrence Consider the following
More informationCS2223 Algorithms D Term 2009 Exam 3 Solutions
CS2223 Algorithms D Term 2009 Exam 3 Solutions May 4, 2009 By Prof. Carolina Ruiz Dept. of Computer Science WPI PROBLEM 1: Asymptoptic Growth Rates (10 points) Let A and B be two algorithms with runtimes
More information