CS 4104 Data and Algorithm Analysis. Recurrence Relations. Modeling Recursive Function Cost. Solving Recurrences. Clifford A. Shaffer.
|
|
- David Melvin Simpson
- 6 years ago
- Views:
Transcription
1 Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2010,2017 by Clifford A. Shaffer Data and Algorithm Analysis Title page Data and Algorithm Analysis Clifford A. Shaffer Spring 2017 Clifford A. Shaffer Department of Computer Science Virginia Tech Blacksburg, Virginia Spring 2017 Copyright c 2010,2017 by Clifford A. Shaffer Data and Algorithm Analysis Spring / 382 Recurrence Relations A (math) function defined in terms of itself. Example: Fibonacci numbers: F(n) = F(n 1) + F(n 2) general case F(1) = F(2) = 1 base cases There are always one or more general cases and one or more base cases. We will use recurrences for time complexity of recursive (computer) functions. General format is T (n) = E(T, n) where E(T, n) is an expression in T and n. T (n) = 2T (n/2) + n Alternately, an upper bound: T (n) E(T, n). Recurrence Relations Recurrence Relations A (math) function defined in terms of itself. Example: Fibonacci numbers: F(n) = F(n 1) + F(n 2) general case F(1) = F(2) = 1 base cases There are always one or more general cases and one or more base cases. We will use recurrences for time complexity of recursive (computer) functions. General format is T (n) = E(T, n) where E(T, n) is an expression in T and n. T (n) = 2T (n/2) + n Alternately, an upper bound: T (n) E(T, n). We won t spend a lot of time on techniques... just enough to be able to use them. Data and Algorithm Analysis Spring / 382 Modeling Recursive Function Cost How does Binary Search work? Look at the middle element, then do binary search on one side. T (n) = T (n/2) + 1; f (1) = 1; no note Modeling Recursive Function Cost Modeling Recursive Function Cost How does Binary Search work? Look at the middle element, then do binary search on one side. T (n) = T (n/2) + 1; f (1) = 1; Mergesort: T (n) = 2T (n/2) + O(n) TOH: T (n) = 2T (n 1) + 1; T (1) = 0 Insertion Sort or Bubble Sort: Typically think of these as double for loops that naturally yeild a summation Can think of them recursively: T (n) = T (n 1) + O(n); T (1) = 1 Mergesort: T (n) = 2T (n/2) + O(n) TOH: T (n) = 2T (n 1) + 1; T (1) = 0 Insertion Sort or Bubble Sort: Typically think of these as double for loops that naturally yeild a summation Can think of them recursively: T (n) = T (n 1) + O(n); T (1) = 1 Data and Algorithm Analysis Spring / 382 Solving Recurrences Solving Recurrences Solving Recurrences We would like to find a closed form solution for T (n) such that: T (n) = Θ(f (n)) Methods: Guess (and test) a solution Expand recurrence Theorems We would like to find a closed form solution for T (n) such that: T (n) = Θ(f (n)) Methods: Guess (and test) a solution Expand recurrence Theorems Note that finding a closed form means that we have f (n) that doesn t include T. Guessing is useful for finding an asymptotic solution. Use induction to prove the guess correct. Data and Algorithm Analysis Spring / 382
2 Guessing n 2 Note that T is defined only for powers of 2. Guess a solution: T (n) c 1 n 3 = f (n) implies that c 1 7 Inductively, assume T (n/2) f (n/2). 2c 1 (n/2) 3 + 5n 2 c 1 (n 3 /4) + 5n 2 c 1 n 3 if c 1 20/3. Guessing For Big-oh, not many choices in what to guess = 7 Guessing n 2 Note that T is defined only for powers of 2. Guess a solution: T (n) c1n 3 = f (n) implies that c1 7 Inductively, assume T (n/2) f (n/2). 2c1(n/2) 3 + 5n 2 c1(n 3 /4) + 5n 2 c1n 3 if c1 20/3. Because n3 + 5n 2 = 20 3 n3 when n = 1, and as n grows, the right side grows even faster. Data and Algorithm Analysis Spring / 382 Therefore, if c1 = 7, a proof by induction yields: T (n) 7n 3 T (n) O(n 3 ) Is this the best possible solution? No - try something tighter. Therefore, if c 1 = 7, a proof by induction yields: T (n) 7n 3 T (n) O(n 3 ) Is this the best possible solution? Data and Algorithm Analysis Spring / 382 Guess again. implies c 2 7. T (n) c 2 n 2 = g(n) Inductively, assume T (n/2) g(n/2). 2c 2 (n/2) 2 + 5n 2 = c 2 (n 2 /2) + 5n 2 c 2 n 2 if c 2 10 Guess again. implies c2 7. T (n) c2n 2 = g(n) Inductively, assume T (n/2) g(n/2). 2c2(n/2) 2 + 5n 2 = c2(n 2 /2) + 5n 2 c2n 2 if c2 10 Therefore, if c2 = 10, T (n) 10n 2. T (n) = O(n 2 ). Is this the best possible upper bound? Because 10 2 n2 + 5n 2 = 10n 2 for n = 1, and the right hand side grows faster. Yes this is best, since T (n) can be as bad as 5n 2. Therefore, if c 2 = 10, T (n) 10n 2. T (n) = O(n 2 ). Is this the best possible upper bound? Data and Algorithm Analysis Spring / 382 Now, reshape the recurrence so that T is defined for all values of n. T (n) 2T ( n/2 ) + 5n 2 n 2 For arbitrary n, let 2 k 1 < n 2 k. We have already shown that T (2 k ) 10(2 k ) 2. T (n) T (2 k ) 10(2 k ) 2 = 10(2 k /n) 2 n 2 10(2) 2 n 2 40n 2 Hence, T (n) = O(n 2 ) for all values of n. Typically, the bound for powers of two generalizes to all n. Data and Algorithm Analysis Spring / 382 Now, reshape the recurrence so that T is defined for all values of n. T (n) 2T ( n/2 ) + 5n 2 n 2 For arbitrary n, let 2 k 1 < n 2 k. We have already shown that T (2 k ) 10(2 k ) 2. T (n) T (2 k ) 10(2 k ) 2 = 10(2 k /n) 2 n 2 10(2) 2 n 2 40n 2 Hence, T (n) = O(n 2 ) for all values of n. Typically, the bound for powers of two generalizes to all n.
3 k 1 Expanding Recurrences (Intro) Expanding Recurrences (Intro) Expanding Recurrences (Intro) Take a look at the simple example slideshows at OpenDSA. no note Take a look at the simple example slideshows at OpenDSA. Data and Algorithm Analysis Spring / 382 Expanding Recurrences Expanding Recurrences Expanding Recurrences Usually, start with equality version of recurrence. Assume n is a power of 2; n = 2 k. Usually, start with equality version of recurrence. Assume n is a power of 2; n = 2 k. Data and Algorithm Analysis Spring / 382 Expanding Recurrences (cont) Expanding Recurrences (cont) Expanding Recurrences (cont) = 2(2T (n/4) + 5(n/2) 2 ) + 5n 2 = 2(2(2T (n/8) + 5(n/4) 2 ) + 5(n/2) 2 ) + 5n 2 = 2 k T (1) + 2 k 1 5(n/2 k 1 ) k 2 5(n/2 k 2 ) (n/2) 2 + 5n 2 k 1 = 7n + 5 n 2 /2 i = 7n + 5n2 1/2 i = 7n + 5n 2 (2 1/2 k 1 ) = 7n + 5n 2 (2 2/n). = 2(2T (n/4) + 5(n/2) 2 ) + 5n 2 = 2(2(2T (n/8) + 5(n/4) 2 ) + 5(n/2) 2 ) + 5n 2 = 2 k T (1) + 2 k 1 5(n/2 k 1 ) k 2 5(n/2 k 2 ) (n/2) 2 + 5n 2 k 1 k 1 = 7n + 5 n 2 /2 i = 7n + 5n 2 1/2 i = 7n + 5n 2 (2 1/2 k 1 ) = 7n + 5n 2 (2 2/n). This it the exact solution for powers of 2. T (n) = Θ(n 2 ). This it the exact solution for powers of 2. T (n) = Θ(n 2 ). Data and Algorithm Analysis Spring / 382 These have the form: T (n) = at (n/b) + cn k T (1) = c... where a, b, c, k are constants. A problem of size n is divided into a subproblems of size n/b, while cn k is the amount of work needed to combine the solutions. These have the form: T (n) = at (n/b) + cn k T (1) = c... where a, b, c, k are constants. A problem of size n is divided into a subproblems of size n/b, while cn k is the amount of work needed to combine the solutions. Data and Algorithm Analysis Spring / 382
4 (cont) Expand the sum; n = b m. T (n) = a(at (n/b 2 ) + c(n/b) k ) + cn k = a m T (1) + a m 1 c(n/b m 1 ) k + + ac(n/b) k + cn k m = ca m (b k /a) i a m = a log b n = n log b a The summation is a geometric series whose sum depends on the ratio r = b k /a. There are 3 cases. Data and Algorithm Analysis Spring / 382 (cont) n = b m m = log b n. Set a = b log b a. Switch order of logs, giving (b log b n ) log b a = n log b a. (cont) Expand the sum; n = b m. T (n) = a(at (n/b 2 ) + c(n/b) k ) + cn k = a m T (1) + a m 1 c(n/b m 1 ) k + + ac(n/b) k + cn k m = ca m (b k /a) i a m = a log b n = n log b a The summation is a geometric series whose sum depends on the ratio r = b k /a. There are 3 cases. D & C Recurrences (cont) D & C Recurrences (cont) D & C Recurrences (cont) (1) r < 1. (2) r = 1. i r < 1/(1 r), a constant. T (n) = Θ(a m ) = Θ(n log b a ). r i = m + 1 = log b n + 1 (1) r < 1. r i < 1/(1 r), a constant. When r = 1, since r = b k /a = 1, we get a = b k. Recall that k = log b a. T (n) = Θ(n log b a log n) = Θ(n k log n) T (n) = Θ(a m ) = Θ(n log b a ). (2) r = 1. r i = m + 1 = log b n + 1 T (n) = Θ(n log b a log n) = Θ(n k log n) Data and Algorithm Analysis Spring / 382 D & C Recurrences (Case 3) D & C Recurrences (Case 3) (3) r > 1. D & C Recurrences (Case 3) So, from T (n) = ca m r i, r i = r m+1 1 r 1 = Θ(r m ) T (n) = Θ(a m r m ) = Θ(a m (b k /a) m ) = Θ(b km ) = Θ(n k ) (3) r > 1. So, from T (n) = ca m r i, r i = r m+1 1 r 1 = Θ(r m ) T (n) = Θ(a m r m ) = Θ(a m (b k /a) m ) = Θ(b km ) = Θ(n k ) Data and Algorithm Analysis Spring / 382 Summary Summary Theorem 3.4: T (n) = Apply the theorem: T (n) = 3T (n/5) + 8n 2. a = 3, b = 5, c = 8, k = 2. b k /a = 25/3. Summary Θ(n log b a ) Θ(n k log n) Θ(n k ) if a > b k if a = b k if a < b k Case (3) holds: T (n) = Θ(n 2 ). Theorem 3.4: Θ(n log b a ) T (n) = Θ(n k log n) Θ(n k ) if a > b k if a = b k if a < b k We simplify by approximating summations. Apply the theorem: T (n) = 3T (n/5) + 8n 2. a = 3, b = 5, c = 8, k = 2. b k /a = 25/3. Case (3) holds: T (n) = Θ(n 2 ). Data and Algorithm Analysis Spring / 382
5 Examples Examples Examples Mergesort: T (n) = 2T (n/2) + n. 2 1 /2 = 1, so T (n) = Θ(n log n). Binary search: T (n) = T (n/2) /1 = 1, so T (n) = Θ(log n). Insertion sort: T (n) = T (n 1) + n. Can t apply the theorem. Sorry! Standard Matrix Multiply (recursively): T (n) = 8T (n/2) + n /8 = 1/2 so T (n) = Θ(n log 2 8 ) = Θ(n 3 ). Mergesort: T (n) = 2T (n/2) + n. 2 1 /2 = 1, so T (n) = Θ(n log n). Binary search: T (n) = T (n/2) /1 = 1, so T (n) = Θ(log n). Insertion sort: T (n) = T (n 1) + n. Can t apply the theorem. Sorry! Standard Matrix Multiply (recursively): T (n) = 8T (n/2) + n /8 = 1/2 so T (n) = Θ(n log 2 8 ) = Θ(n 3 ). [ c11 c 12 c 21 c 22 ] [ ] [ ] a11 a = 12 b11 b 12 a 21 a 22 b 21 b 22 In the straightforward implementation, 2 2 case is: c 11 = a 11 b 11 + a 12 b 21 c 12 = a 11 b 12 + a 12 b 22 c 21 = a 21 b 11 + a 22 b 21 c 22 = a 21 b 12 + a 22 b 22 So the recursion is 8 calls of half size, and the additions take Θ(n 2 ) work. Data and Algorithm Analysis Spring / 382 Searching Searching Searching Assumptions for search problems: Target is well defined. Target is fixed. Search domain is finite. We (can) remember all information gathered during search. We search for a record with a key. Assumptions for search problems: Target is well defined. Target is fixed. Search domain is finite. We (can) remember all information gathered during search. Well defined: We recognize a hit or miss. Fixed: The target doesn t move during the life of the search. We often choose not to remember information. For example, sequential search does not remember the values seen already. We search for a record with a key. Data and Algorithm Analysis Spring / 382 A Search Model (1) Problem: Given: A list L, of n elements A search key X Solve: Identify one element in L which has key value X, if any exist. Model: The key values for elements in L are unique. One comparison determines <, =, >. Comparison is our only way to find ordering information. Every comparison costs the same. A Search Model (1) A Search Model (1) Problem: Given: A list L, of n elements A search key X Solve: Identify one element in L which has key value X, if any exist. Model: The key values for elements in L are unique. One comparison determines <, =, >. Comparison is our only way to find ordering information. Every comparison costs the same. What if the key values are not unique? Probably the cost goes down, not up. This is an assumption for analysis, not for implementation. We would have a slightly different model (though no asymptotic change in cost) if our only comparison test was <. We would have a very different model if our only comparison was = /. A comparison-based model. String data might require comparisons with very different costs. Data and Algorithm Analysis Spring / 382 A Search Model (2) A Search Model (2) A Search Model (2) Goal: Solve the problem using the minimum number of comparisons. Cost model: Number of comparisons. (Implication) Access to every item in L costs the same (array). Is this a reasonable model and goal? Goal: Solve the problem using the minimum number of comparisons. Cost model: Number of comparisons. (Implication) Access to every item in L costs the same (array). Is this a reasonable model and goal? We are assuming that the # of comparisons is proportional to runtime. Might not always share an array (assumption that all accesses are equal). For example, linked lists. Or data on disk, or across network. We assume there is no relationship between value X and its position. Data and Algorithm Analysis Spring / 382
6 Linear Search General algorithm strategy: Reduce the problem. Compare X to the first element. If not done, then solve the problem for n 1 elements. Position linear_search(l, lower, upper, X) { if L[lower] = X then return lower; else if lower = upper then return -1; else return linear_search(l, lower+1, upper, X); } Linear Search Linear Search General algorithm strategy: Reduce the problem. Compare X to the first element. If not done, then solve the problem for n 1 elements. Position linear_search(l, lower, upper, X) { if L[lower] = X then return lower; else if lower = upper then return -1; else return linear_search(l, lower+1, upper, X); } What equation represents the worst case cost? Warning: We are using this simple, familiar algorithm as an illustration of how to do full, formal analysis. This includes some recurrence solving techniques, and attention to lower bounds. Cost given on next slide. What equation represents the worst case cost? Data and Algorithm Analysis Spring / 382 Worst Cost Upper Bound Worst Cost Upper Bound Worst Cost Upper Bound { 1 n = 1 f (n) = f (n 1) + 1 n > 1 Reasonable to guess that f (n) = n. Prove by induction: Basis step: f (1) = 1, so f (n) = n when n = 1. Induction hypothesis: For k < n, f (k) = k. Induction step: From recurrence, = (n 1) + 1 = n { 1 n = 1 f (n) = f (n 1) + 1 n > 1 Reasonable to guess that f (n) = n. Prove by induction: Basis step: f (1) = 1, so f (n) = n when n = 1. Induction hypothesis: For k < n, f (k) = k. Induction step: From recurrence, = (n 1) + 1 = n Thus, the worst case cost for n elements is linear. Induction is great for verifying a hypothesis. Thus, the worst case cost for n elements is linear. Induction is great for verifying a hypothesis. Data and Algorithm Analysis Spring / 382 Approach #2 What if we couldn t guess a solution? Try: Substitute and Guess. Iterate a few steps of the recurrence, and look for a summation. = {f (n 2) + 1} + 1 = {{f (n 3) + 1} + 1} + 1} Approach #2 Replace i with n 1. Alternative: Recognize f (n) = f (1 n + i=2 1. Approach #2 What if we couldn t guess a solution? Try: Substitute and Guess. Iterate a few steps of the recurrence, and look for a summation. = {f (n 2) + 1} + 1 = {{f (n 3) + 1} + 1} + 1} Now what? Guess f (n) = f (n i) + i. When do we stop? When we reach a value for f that we know. f (n) = f (n (n 1)) + n 1 = f (1) + n 1 = n Now, go back and test the guess using induction. Now what? Guess f (n) = f (n i) + i. When do we stop? When we reach a value for f that we know. f (n) = f (n (n 1)) + n 1 = f (1) + n 1 = n Now, go back and test the guess using induction. Data and Algorithm Analysis Spring / 382 Approach #3 Guess and Test: Guess the form of the solution, then solve the resulting equations. Guess: f (n) is linear. f (n) = rn + s for some r, s. What do we know? f (1) = r 1 + s = r + s = 1. f (n) = r n + s = r (n 1) + s + 1. Solving these two simultaneous equations, r = 1, s = 0. Final form of guess: f (n) = n. Now, prove using induction. Data and Algorithm Analysis Spring / 382 Approach #3 Often, f (0) is easier. Or maybe f (2). Approach #3 Guess and Test: Guess the form of the solution, then solve the resulting equations. Guess: f (n) is linear. f (n) = rn + s for some r, s. What do we know? f (1) = r 1 + s = r + s = 1. f (n) = r n + s = r (n 1) + s + 1. Solving these two simultaneous equations, r = 1, s = 0. Final form of guess: f (n) = n. Now, prove using induction. By definition,, so r n = r (n 1) + 1. So rn + s = rn r + s + 1. s = s r + 1 r 1 = 0 Since. Why is this a guess and not a proof? Because all we did is show that our model passes through two points that the real curve also passes through. If the curve really is linear, 2 points is all that we need. But, we need to prove that it is linear.
7 Lower Bound on the Problem (1) comparing X with L[n]. We can feed the algorithm an input with X in position n. Lower Bound on the Problem (1) Lower Bound on the Problem (1) comparing X with L[n]. We can feed the algorithm an input with X in position n. Be careful about assumptions on how an algorithm might (must) behave. After all, where do new, clever algorithms come from? From different behavior than was previously assumed! Data and Algorithm Analysis Spring / 382 Fixing the Proof (1) Fixing the Proof (1) Fixing the Proof (1) Error #1: An algorithm need not consistently skip position n. Itcould, for example, work from right to left. Fix: Generalize to skip some other position i. Error #1: An algorithm need not consistently skip position n. Itcould, for example, work from right to left. Fix: Generalize to skip some other position i. Data and Algorithm Analysis Spring / 382 Lower Bound on the Problem (2) Lower Bound on the Problem (2) Lower Bound on the Problem (2) comparing X with L[i] for some value i. We can feed the algorithm an input with X in position i. comparing X with L[i] for some value i. We can feed the algorithm an input with X in position i. Data and Algorithm Analysis Spring / 382
/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17
601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17 2.1 Notes Homework 1 will be released today, and is due a week from today by the beginning
More informationCS 577 Introduction to Algorithms: Strassen s Algorithm and the Master Theorem
CS 577 Introduction to Algorithms: Jin-Yi Cai University of Wisconsin Madison In the last class, we described InsertionSort and showed that its worst-case running time is Θ(n 2 ). Check Figure 2.2 for
More informationCOMP Analysis of Algorithms & Data Structures
COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 10, 2018 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. 1 /
More informationCOMP Analysis of Algorithms & Data Structures
COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 14, 2019 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. COMP
More informationb + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d
CS161, Lecture 4 Median, Selection, and the Substitution Method Scribe: Albert Chen and Juliana Cook (2015), Sam Kim (2016), Gregory Valiant (2017) Date: January 23, 2017 1 Introduction Last lecture, we
More informationLecture 3. Big-O notation, more recurrences!!
Lecture 3 Big-O notation, more recurrences!! Announcements! HW1 is posted! (Due Friday) See Piazza for a list of HW clarifications First recitation section was this morning, there s another tomorrow (same
More informationAnalysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort
Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Xi Chen Columbia University We continue with two more asymptotic notation: o( ) and ω( ). Let f (n) and g(n) are functions that map
More informationAsymptotic Analysis and Recurrences
Appendix A Asymptotic Analysis and Recurrences A.1 Overview We discuss the notion of asymptotic analysis and introduce O, Ω, Θ, and o notation. We then turn to the topic of recurrences, discussing several
More informationCS 310 Advanced Data Structures and Algorithms
CS 310 Advanced Data Structures and Algorithms Runtime Analysis May 31, 2017 Tong Wang UMass Boston CS 310 May 31, 2017 1 / 37 Topics Weiss chapter 5 What is algorithm analysis Big O, big, big notations
More informationCIS 121. Analysis of Algorithms & Computational Complexity. Slides based on materials provided by Mary Wootters (Stanford University)
CIS 121 Analysis of Algorithms & Computational Complexity Slides based on materials provided by Mary Wootters (Stanford University) Today Sorting: InsertionSort vs MergeSort Analyzing the correctness of
More informationMA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3
MA008 p.1/37 MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3 Dr. Markus Hagenbuchner markus@uow.edu.au. MA008 p.2/37 Exercise 1 (from LN 2) Asymptotic Notation When constants appear in exponents
More informationMergesort and Recurrences (CLRS 2.3, 4.4)
Mergesort and Recurrences (CLRS 2.3, 4.4) We saw a couple of O(n 2 ) algorithms for sorting. Today we ll see a different approach that runs in O(n lg n) and uses one of the most powerful techniques for
More informationCS 4104 Data and Algorithm Analysis. CS4014: What You Need to Already Know. CS4104: What We Will Do. Introduction to Problem Solving (1)
Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 010,017 by Clifford A. Shaffer Data and Algorithm Analysis Title page Data and Algorithm Analysis Clifford A. Shaffer Spring
More informationCS173 Running Time and Big-O. Tandy Warnow
CS173 Running Time and Big-O Tandy Warnow CS 173 Running Times and Big-O analysis Tandy Warnow Today s material We will cover: Running time analysis Review of running time analysis of Bubblesort Review
More informationCS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms
CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions
More informationComputational Complexity - Pseudocode and Recursions
Computational Complexity - Pseudocode and Recursions Nicholas Mainardi 1 Dipartimento di Elettronica e Informazione Politecnico di Milano nicholas.mainardi@polimi.it June 6, 2018 1 Partly Based on Alessandro
More informationCS 2110: INDUCTION DISCUSSION TOPICS
CS 110: INDUCTION DISCUSSION TOPICS The following ideas are suggestions for how to handle your discussion classes. You can do as much or as little of this as you want. You can either present at the board,
More informationThe maximum-subarray problem. Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm:
The maximum-subarray problem Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm: Brute force algorithm: At best, θ(n 2 ) time complexity 129 Can we do divide
More information5 + 9(10) + 3(100) + 0(1000) + 2(10000) =
Chapter 5 Analyzing Algorithms So far we have been proving statements about databases, mathematics and arithmetic, or sequences of numbers. Though these types of statements are common in computer science,
More informationComputational Complexity. This lecture. Notes. Lecture 02 - Basic Complexity Analysis. Tom Kelsey & Susmit Sarkar. Notes
Computational Complexity Lecture 02 - Basic Complexity Analysis Tom Kelsey & Susmit Sarkar School of Computer Science University of St Andrews http://www.cs.st-andrews.ac.uk/~tom/ twk@st-andrews.ac.uk
More informationSolving recurrences. Frequently showing up when analysing divide&conquer algorithms or, more generally, recursive algorithms.
Solving recurrences Frequently showing up when analysing divide&conquer algorithms or, more generally, recursive algorithms Example: Merge-Sort(A, p, r) 1: if p < r then 2: q (p + r)/2 3: Merge-Sort(A,
More informationAnalysis of Algorithms - Using Asymptotic Bounds -
Analysis of Algorithms - Using Asymptotic Bounds - Andreas Ermedahl MRTC (Mälardalens Real-Time Research Center) andreas.ermedahl@mdh.se Autumn 004 Rehersal: Asymptotic bounds Gives running time bounds
More informationMethods for solving recurrences
Methods for solving recurrences Analyzing the complexity of mergesort The merge function Consider the following implementation: 1 int merge ( int v1, int n1, int v, int n ) { 3 int r = malloc ( ( n1+n
More informationA design paradigm. Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/ EECS 3101
A design paradigm Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/17 112 Multiplying complex numbers (from Jeff Edmonds slides) INPUT: Two pairs of integers, (a,b),
More informationAn analogy from Calculus: limits
COMP 250 Fall 2018 35 - big O Nov. 30, 2018 We have seen several algorithms in the course, and we have loosely characterized their runtimes in terms of the size n of the input. We say that the algorithm
More informationData Structures and Algorithms Chapter 3
Data Structures and Algorithms Chapter 3 1. Divide and conquer 2. Merge sort, repeated substitutions 3. Tiling 4. Recurrences Recurrences Running times of algorithms with recursive calls can be described
More informationWhat we have learned What is algorithm Why study algorithm The time and space efficiency of algorithm The analysis framework of time efficiency Asympt
Lecture 3 The Analysis of Recursive Algorithm Efficiency What we have learned What is algorithm Why study algorithm The time and space efficiency of algorithm The analysis framework of time efficiency
More informationDivide-and-conquer: Order Statistics. Curs: Fall 2017
Divide-and-conquer: Order Statistics Curs: Fall 2017 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers.
More informationdata structures and algorithms lecture 2
data structures and algorithms 2018 09 06 lecture 2 recall: insertion sort Algorithm insertionsort(a, n): for j := 2 to n do key := A[j] i := j 1 while i 1 and A[i] > key do A[i + 1] := A[i] i := i 1 A[i
More informationData structures Exercise 1 solution. Question 1. Let s start by writing all the functions in big O notation:
Data structures Exercise 1 solution Question 1 Let s start by writing all the functions in big O notation: f 1 (n) = 2017 = O(1), f 2 (n) = 2 log 2 n = O(n 2 ), f 3 (n) = 2 n = O(2 n ), f 4 (n) = 1 = O
More informationCS 4407 Algorithms Lecture 3: Iterative and Divide and Conquer Algorithms
CS 4407 Algorithms Lecture 3: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions
More informationAlgorithms. Quicksort. Slide credit: David Luebke (Virginia)
1 Algorithms Quicksort Slide credit: David Luebke (Virginia) Sorting revisited We have seen algorithms for sorting: INSERTION-SORT, MERGESORT More generally: given a sequence of items Each item has a characteristic
More informationComputational Complexity
Computational Complexity S. V. N. Vishwanathan, Pinar Yanardag January 8, 016 1 Computational Complexity: What, Why, and How? Intuitively an algorithm is a well defined computational procedure that takes
More informationQuiz 1 Solutions. Problem 2. Asymptotics & Recurrences [20 points] (3 parts)
Introduction to Algorithms October 13, 2010 Massachusetts Institute of Technology 6.006 Fall 2010 Professors Konstantinos Daskalakis and Patrick Jaillet Quiz 1 Solutions Quiz 1 Solutions Problem 1. We
More informationWhen we use asymptotic notation within an expression, the asymptotic notation is shorthand for an unspecified function satisfying the relation:
CS 124 Section #1 Big-Oh, the Master Theorem, and MergeSort 1/29/2018 1 Big-Oh Notation 1.1 Definition Big-Oh notation is a way to describe the rate of growth of functions. In CS, we use it to describe
More informationTaking Stock. IE170: Algorithms in Systems Engineering: Lecture 3. Θ Notation. Comparing Algorithms
Taking Stock IE170: Algorithms in Systems Engineering: Lecture 3 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University January 19, 2007 Last Time Lots of funky math Playing
More informationBig O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013
/4/3 Administrative Big O David Kauchak cs3 Spring 3 l Assignment : how d it go? l Assignment : out soon l CLRS code? l Videos Insertion-sort Insertion-sort Does it terminate? /4/3 Insertion-sort Loop
More informationDivide and Conquer CPE 349. Theresa Migler-VonDollen
Divide and Conquer CPE 349 Theresa Migler-VonDollen Divide and Conquer Divide and Conquer is a strategy that solves a problem by: 1 Breaking the problem into subproblems that are themselves smaller instances
More informationWhen we use asymptotic notation within an expression, the asymptotic notation is shorthand for an unspecified function satisfying the relation:
CS 124 Section #1 Big-Oh, the Master Theorem, and MergeSort 1/29/2018 1 Big-Oh Notation 1.1 Definition Big-Oh notation is a way to describe the rate of growth of functions. In CS, we use it to describe
More informationRecurrences COMP 215
Recurrences COMP 215 Analysis of Iterative Algorithms //return the location of the item matching x, or 0 if //no such item is found. index SequentialSearch(keytype[] S, in, keytype x) { index location
More informationCSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2019
CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis Ruth Anderson Winter 2019 Today Algorithm Analysis What do we care about? How to compare two algorithms Analyzing Code Asymptotic Analysis
More informationAlgorithms Design & Analysis. Analysis of Algorithm
Algorithms Design & Analysis Analysis of Algorithm Review Internship Stable Matching Algorithm 2 Outline Time complexity Computation model Asymptotic notions Recurrence Master theorem 3 The problem of
More informationCSC236 Week 4. Larry Zhang
CSC236 Week 4 Larry Zhang 1 Announcements PS2 due on Friday This week s tutorial: Exercises with big-oh PS1 feedback People generally did well Writing style need to be improved. This time the TAs are lenient,
More informationDivide and Conquer. Arash Rafiey. 27 October, 2016
27 October, 2016 Divide the problem into a number of subproblems Divide the problem into a number of subproblems Conquer the subproblems by solving them recursively or if they are small, there must be
More informationBig , and Definition Definition
Big O, Ω, and Θ Big-O gives us only a one-way comparison; if f is O(g) then g eventually is bigger than f from that point on, but in fact f could be very small in comparison. Example; 3n is O(2 2n ). We
More informationCS 161 Summer 2009 Homework #2 Sample Solutions
CS 161 Summer 2009 Homework #2 Sample Solutions Regrade Policy: If you believe an error has been made in the grading of your homework, you may resubmit it for a regrade. If the error consists of more than
More informationDesign and Analysis of Algorithms
CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 5: Divide and Conquer (Part 2) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ A Lower Bound on Convex Hull Lecture 4 Task: sort the
More informationAnalysis of Algorithm Efficiency. Dr. Yingwu Zhu
Analysis of Algorithm Efficiency Dr. Yingwu Zhu Measure Algorithm Efficiency Time efficiency How fast the algorithm runs; amount of time required to accomplish the task Our focus! Space efficiency Amount
More informationUCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis
UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis Lecture 14 Class URL: http://vlsicad.ucsd.edu/courses/cse21-s14/ Lecture 14 Notes Goals for this week Big-O complexity
More informationAlgorithms, Design and Analysis. Order of growth. Table 2.1. Big-oh. Asymptotic growth rate. Types of formulas for basic operation count
Types of formulas for basic operation count Exact formula e.g., C(n) = n(n-1)/2 Algorithms, Design and Analysis Big-Oh analysis, Brute Force, Divide and conquer intro Formula indicating order of growth
More informationAlgorithm Analysis Recurrence Relation. Chung-Ang University, Jaesung Lee
Algorithm Analysis Recurrence Relation Chung-Ang University, Jaesung Lee Recursion 2 Recursion 3 Recursion in Real-world Fibonacci sequence = + Initial conditions: = 0 and = 1. = + = + = + 0, 1, 1, 2,
More informationGrade 11/12 Math Circles Fall Nov. 5 Recurrences, Part 2
1 Faculty of Mathematics Waterloo, Ontario Centre for Education in Mathematics and Computing Grade 11/12 Math Circles Fall 2014 - Nov. 5 Recurrences, Part 2 Running time of algorithms In computer science,
More informationA SUMMARY OF RECURSION SOLVING TECHNIQUES
A SUMMARY OF RECURSION SOLVING TECHNIQUES KIMMO ERIKSSON, KTH These notes are meant to be a complement to the material on recursion solving techniques in the textbook Discrete Mathematics by Biggs. In
More informationData Structures and Algorithms Chapter 3
1 Data Structures and Algorithms Chapter 3 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples
More informationCSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2018
CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis Ruth Anderson Winter 2018 Today Algorithm Analysis What do we care about? How to compare two algorithms Analyzing Code Asymptotic Analysis
More informationCS 5114: Theory of Algorithms. Tractable Problems. Tractable Problems (cont) Decision Problems. Clifford A. Shaffer. Spring 2014
Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2014 by Clifford A. Shaffer : Theory of Algorithms Title page : Theory of Algorithms Clifford A. Shaffer Spring 2014 Clifford
More informationAlgorithm efficiency analysis
Algorithm efficiency analysis Mădălina Răschip, Cristian Gaţu Faculty of Computer Science Alexandru Ioan Cuza University of Iaşi, Romania DS 2017/2018 Content Algorithm efficiency analysis Recursive function
More informationCSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2018
CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis Ruth Anderson Winter 2018 Today Algorithm Analysis What do we care about? How to compare two algorithms Analyzing Code Asymptotic Analysis
More informationTopic 17. Analysis of Algorithms
Topic 17 Analysis of Algorithms Analysis of Algorithms- Review Efficiency of an algorithm can be measured in terms of : Time complexity: a measure of the amount of time required to execute an algorithm
More informationGrowth of Functions (CLRS 2.3,3)
Growth of Functions (CLRS 2.3,3) 1 Review Last time we discussed running time of algorithms and introduced the RAM model of computation. Best-case running time: the shortest running time for any input
More informationIn-Class Soln 1. CS 361, Lecture 4. Today s Outline. In-Class Soln 2
In-Class Soln 1 Let f(n) be an always positive function and let g(n) = f(n) log n. Show that f(n) = o(g(n)) CS 361, Lecture 4 Jared Saia University of New Mexico For any positive constant c, we want to
More informationCS2223 Algorithms D Term 2009 Exam 3 Solutions
CS2223 Algorithms D Term 2009 Exam 3 Solutions May 4, 2009 By Prof. Carolina Ruiz Dept. of Computer Science WPI PROBLEM 1: Asymptoptic Growth Rates (10 points) Let A and B be two algorithms with runtimes
More informationSolving Recurrences. 1. Express the running time (or use of some other resource) as a recurrence.
Solving Recurrences Recurrences and Recursive Code Many (perhaps most) recursive algorithms fall into one of two categories: tail recursion and divide-andconquer recursion. We would like to develop some
More informationDivide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,
Divide and Conquer CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Merging sorted lists: WHAT Given two sorted lists a 1 a 2 a 3 a k b 1 b 2 b 3 b
More informationAdvanced Counting Techniques. Chapter 8
Advanced Counting Techniques Chapter 8 Chapter Summary Applications of Recurrence Relations Solving Linear Recurrence Relations Homogeneous Recurrence Relations Nonhomogeneous Recurrence Relations Divide-and-Conquer
More informationReview Of Topics. Review: Induction
Review Of Topics Asymptotic notation Solving recurrences Sorting algorithms Insertion sort Merge sort Heap sort Quick sort Counting sort Radix sort Medians/order statistics Randomized algorithm Worst-case
More information1 Substitution method
Recurrence Relations we have discussed asymptotic analysis of algorithms and various properties associated with asymptotic notation. As many algorithms are recursive in nature, it is natural to analyze
More informationi=1 i B[i] B[i] + A[i, j]; c n for j n downto i + 1 do c n i=1 (n i) C[i] C[i] + A[i, j]; c n
Fundamental Algorithms Homework #1 Set on June 25, 2009 Due on July 2, 2009 Problem 1. [15 pts] Analyze the worst-case time complexity of the following algorithms,and give tight bounds using the Theta
More informationCOL106: Data Structures and Algorithms (IIT Delhi, Semester-II )
1 Solve the following recurrence relations giving a Θ bound for each of the cases 1 : (a) T (n) = 2T (n/3) + 1; T (1) = 1 (Assume n is a power of 3) (b) T (n) = 5T (n/4) + n; T (1) = 1 (Assume n is a power
More informationMath 3361-Modern Algebra Lecture 08 9/26/ Cardinality
Math 336-Modern Algebra Lecture 08 9/26/4. Cardinality I started talking about cardinality last time, and you did some stuff with it in the Homework, so let s continue. I said that two sets have the same
More informationNotes for Recitation 14
6.04/18.06J Mathematics for Computer Science October 4, 006 Tom Leighton and Marten van Dijk Notes for Recitation 14 1 The Akra-Bazzi Theorem Theorem 1 (Akra-Bazzi, strong form). Suppose that: is defined
More informationDivide and Conquer. Recurrence Relations
Divide and Conquer Recurrence Relations Divide-and-Conquer Strategy: Break up problem into parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. 2 MergeSort Mergesort.
More informationIntroduction to Divide and Conquer
Introduction to Divide and Conquer Sorting with O(n log n) comparisons and integer multiplication faster than O(n 2 ) Periklis A. Papakonstantinou York University Consider a problem that admits a straightforward
More informationAnalysis of Algorithms [Reading: CLRS 2.2, 3] Laura Toma, csci2200, Bowdoin College
Analysis of Algorithms [Reading: CLRS 2.2, 3] Laura Toma, csci2200, Bowdoin College Why analysis? We want to predict how the algorithm will behave (e.g. running time) on arbitrary inputs, and how it will
More informationSolving Recurrences. 1. Express the running time (or use of some other resource) as a recurrence.
1 Recurrences and Recursive Code Solving Recurrences Many (perhaps most) recursive algorithms fall into one of two categories: tail recursion and divide-andconquer recursion. We would like to develop some
More informationCS483 Design and Analysis of Algorithms
CS483 Design and Analysis of Algorithms Chapter 2 Divide and Conquer Algorithms Instructor: Fei Li lifei@cs.gmu.edu with subject: CS483 Office hours: Room 5326, Engineering Building, Thursday 4:30pm -
More informationOmega notation. Transitivity etc.
Omega notation Big-Omega: Lecture 2, Sept. 25, 2014 f () n (()) g n const cn, s.t. n n : cg() n f () n Small-omega: 0 0 0 f () n (()) g n const c, n s.t. n n : cg() n f () n 0 0 0 Intuition (works most
More informationRecurrence Relations
Recurrence Relations Analysis Tools S.V. N. (vishy) Vishwanathan University of California, Santa Cruz vishy@ucsc.edu January 15, 2016 S.V. N. Vishwanathan (UCSC) CMPS101 1 / 29 Recurrences Outline 1 Recurrences
More informationLecture 2: Asymptotic Notation CSCI Algorithms I
Lecture 2: Asymptotic Notation CSCI 700 - Algorithms I Andrew Rosenberg September 2, 2010 Last Time Review Insertion Sort Analysis of Runtime Proof of Correctness Today Asymptotic Notation Its use in analyzing
More informationDivide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch] Divide and Conquer Paradigm An important general technique for designing algorithms: divide problem into subproblems recursively
More informationAsymptotic Analysis 1
Asymptotic Analysis 1 Last week, we discussed how to present algorithms using pseudocode. For example, we looked at an algorithm for singing the annoying song 99 Bottles of Beer on the Wall for arbitrary
More informationData Structures and Algorithms CMPSC 465
Data Structures and Algorithms CMPSC 465 LECTURE 9 Solving recurrences Substitution method Adam Smith S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson Review question Draw
More informationRecurrence Relations
Recurrence Relations Winter 2017 Recurrence Relations Recurrence Relations A recurrence relation for the sequence {a n } is an equation that expresses a n in terms of one or more of the previous terms
More informationLecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018
CS17 Integrated Introduction to Computer Science Klein Contents Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 1 Tree definitions 1 2 Analysis of mergesort using a binary tree 1 3 Analysis of
More informationDivide and Conquer Algorithms. CSE 101: Design and Analysis of Algorithms Lecture 14
Divide and Conquer Algorithms CSE 101: Design and Analysis of Algorithms Lecture 14 CSE 101: Design and analysis of algorithms Divide and conquer algorithms Reading: Sections 2.3 and 2.4 Homework 6 will
More informationProblem Set 1. CSE 373 Spring Out: February 9, 2016
Problem Set 1 CSE 373 Spring 2016 Out: February 9, 2016 1 Big-O Notation Prove each of the following using the definition of big-o notation (find constants c and n 0 such that f(n) c g(n) for n > n o.
More informationLecture 3: Big-O and Big-Θ
Lecture 3: Big-O and Big-Θ COSC4: Algorithms and Data Structures Brendan McCane Department of Computer Science, University of Otago Landmark functions We saw that the amount of work done by Insertion Sort,
More informationITEC2620 Introduction to Data Structures
ITEC2620 Introduction to Data Structures Lecture 6a Complexity Analysis Recursive Algorithms Complexity Analysis Determine how the processing time of an algorithm grows with input size What if the algorithm
More informationCS Analysis of Recursive Algorithms and Brute Force
CS483-05 Analysis of Recursive Algorithms and Brute Force Instructor: Fei Li Room 443 ST II Office hours: Tue. & Thur. 4:30pm - 5:30pm or by appointments lifei@cs.gmu.edu with subject: CS483 http://www.cs.gmu.edu/
More informationV. Adamchik 1. Recurrences. Victor Adamchik Fall of 2005
V. Adamchi Recurrences Victor Adamchi Fall of 00 Plan Multiple roots. More on multiple roots. Inhomogeneous equations 3. Divide-and-conquer recurrences In the previous lecture we have showed that if the
More informationModule 1: Analyzing the Efficiency of Algorithms
Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu What is an Algorithm?
More informationOutline. 1 Introduction. Merging and MergeSort. 3 Analysis. 4 Reference
Outline Computer Science 331 Sort Mike Jacobson Department of Computer Science University of Calgary Lecture #25 1 Introduction 2 Merging and 3 4 Reference Mike Jacobson (University of Calgary) Computer
More informationLecture 2. More Algorithm Analysis, Math and MCSS By: Sarah Buchanan
Lecture 2 More Algorithm Analysis, Math and MCSS By: Sarah Buchanan Announcements Assignment #1 is posted online It is directly related to MCSS which we will be talking about today or Monday. There are
More informationPractical Session #3 - Recursions
Practical Session #3 - Recursions Substitution method Guess the form of the solution and prove it by induction Iteration Method Convert the recurrence into a summation and solve it Tightly bound a recurrence
More information1 Closest Pair of Points on the Plane
CS 31: Algorithms (Spring 2019): Lecture 5 Date: 4th April, 2019 Topic: Divide and Conquer 3: Closest Pair of Points on a Plane Disclaimer: These notes have not gone through scrutiny and in all probability
More informationUniversity of New Mexico Department of Computer Science. Midterm Examination. CS 361 Data Structures and Algorithms Spring, 2003
University of New Mexico Department of Computer Science Midterm Examination CS 361 Data Structures and Algorithms Spring, 2003 Name: Email: Print your name and email, neatly in the space provided above;
More informationSection Summary. Sequences. Recurrence Relations. Summations. Examples: Geometric Progression, Arithmetic Progression. Example: Fibonacci Sequence
Section 2.4 Section Summary Sequences. Examples: Geometric Progression, Arithmetic Progression Recurrence Relations Example: Fibonacci Sequence Summations Introduction Sequences are ordered lists of elements.
More informationRecursion: Introduction and Correctness
Recursion: Introduction and Correctness CSE21 Winter 2017, Day 7 (B00), Day 4-5 (A00) January 25, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Today s Plan From last time: intersecting sorted lists and
More informationLecture 2. Fundamentals of the Analysis of Algorithm Efficiency
Lecture 2 Fundamentals of the Analysis of Algorithm Efficiency 1 Lecture Contents 1. Analysis Framework 2. Asymptotic Notations and Basic Efficiency Classes 3. Mathematical Analysis of Nonrecursive Algorithms
More informationCS 470/570 Divide-and-Conquer. Format of Divide-and-Conquer algorithms: Master Recurrence Theorem (simpler version)
CS 470/570 Divide-and-Conquer Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively on smaller pieces Combine: Build the
More information