CS 4104 Data and Algorithm Analysis. Recurrence Relations. Modeling Recursive Function Cost. Solving Recurrences. Clifford A. Shaffer.

Size: px
Start display at page:

Download "CS 4104 Data and Algorithm Analysis. Recurrence Relations. Modeling Recursive Function Cost. Solving Recurrences. Clifford A. Shaffer."

Transcription

1 Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2010,2017 by Clifford A. Shaffer Data and Algorithm Analysis Title page Data and Algorithm Analysis Clifford A. Shaffer Spring 2017 Clifford A. Shaffer Department of Computer Science Virginia Tech Blacksburg, Virginia Spring 2017 Copyright c 2010,2017 by Clifford A. Shaffer Data and Algorithm Analysis Spring / 382 Recurrence Relations A (math) function defined in terms of itself. Example: Fibonacci numbers: F(n) = F(n 1) + F(n 2) general case F(1) = F(2) = 1 base cases There are always one or more general cases and one or more base cases. We will use recurrences for time complexity of recursive (computer) functions. General format is T (n) = E(T, n) where E(T, n) is an expression in T and n. T (n) = 2T (n/2) + n Alternately, an upper bound: T (n) E(T, n). Recurrence Relations Recurrence Relations A (math) function defined in terms of itself. Example: Fibonacci numbers: F(n) = F(n 1) + F(n 2) general case F(1) = F(2) = 1 base cases There are always one or more general cases and one or more base cases. We will use recurrences for time complexity of recursive (computer) functions. General format is T (n) = E(T, n) where E(T, n) is an expression in T and n. T (n) = 2T (n/2) + n Alternately, an upper bound: T (n) E(T, n). We won t spend a lot of time on techniques... just enough to be able to use them. Data and Algorithm Analysis Spring / 382 Modeling Recursive Function Cost How does Binary Search work? Look at the middle element, then do binary search on one side. T (n) = T (n/2) + 1; f (1) = 1; no note Modeling Recursive Function Cost Modeling Recursive Function Cost How does Binary Search work? Look at the middle element, then do binary search on one side. T (n) = T (n/2) + 1; f (1) = 1; Mergesort: T (n) = 2T (n/2) + O(n) TOH: T (n) = 2T (n 1) + 1; T (1) = 0 Insertion Sort or Bubble Sort: Typically think of these as double for loops that naturally yeild a summation Can think of them recursively: T (n) = T (n 1) + O(n); T (1) = 1 Mergesort: T (n) = 2T (n/2) + O(n) TOH: T (n) = 2T (n 1) + 1; T (1) = 0 Insertion Sort or Bubble Sort: Typically think of these as double for loops that naturally yeild a summation Can think of them recursively: T (n) = T (n 1) + O(n); T (1) = 1 Data and Algorithm Analysis Spring / 382 Solving Recurrences Solving Recurrences Solving Recurrences We would like to find a closed form solution for T (n) such that: T (n) = Θ(f (n)) Methods: Guess (and test) a solution Expand recurrence Theorems We would like to find a closed form solution for T (n) such that: T (n) = Θ(f (n)) Methods: Guess (and test) a solution Expand recurrence Theorems Note that finding a closed form means that we have f (n) that doesn t include T. Guessing is useful for finding an asymptotic solution. Use induction to prove the guess correct. Data and Algorithm Analysis Spring / 382

2 Guessing n 2 Note that T is defined only for powers of 2. Guess a solution: T (n) c 1 n 3 = f (n) implies that c 1 7 Inductively, assume T (n/2) f (n/2). 2c 1 (n/2) 3 + 5n 2 c 1 (n 3 /4) + 5n 2 c 1 n 3 if c 1 20/3. Guessing For Big-oh, not many choices in what to guess = 7 Guessing n 2 Note that T is defined only for powers of 2. Guess a solution: T (n) c1n 3 = f (n) implies that c1 7 Inductively, assume T (n/2) f (n/2). 2c1(n/2) 3 + 5n 2 c1(n 3 /4) + 5n 2 c1n 3 if c1 20/3. Because n3 + 5n 2 = 20 3 n3 when n = 1, and as n grows, the right side grows even faster. Data and Algorithm Analysis Spring / 382 Therefore, if c1 = 7, a proof by induction yields: T (n) 7n 3 T (n) O(n 3 ) Is this the best possible solution? No - try something tighter. Therefore, if c 1 = 7, a proof by induction yields: T (n) 7n 3 T (n) O(n 3 ) Is this the best possible solution? Data and Algorithm Analysis Spring / 382 Guess again. implies c 2 7. T (n) c 2 n 2 = g(n) Inductively, assume T (n/2) g(n/2). 2c 2 (n/2) 2 + 5n 2 = c 2 (n 2 /2) + 5n 2 c 2 n 2 if c 2 10 Guess again. implies c2 7. T (n) c2n 2 = g(n) Inductively, assume T (n/2) g(n/2). 2c2(n/2) 2 + 5n 2 = c2(n 2 /2) + 5n 2 c2n 2 if c2 10 Therefore, if c2 = 10, T (n) 10n 2. T (n) = O(n 2 ). Is this the best possible upper bound? Because 10 2 n2 + 5n 2 = 10n 2 for n = 1, and the right hand side grows faster. Yes this is best, since T (n) can be as bad as 5n 2. Therefore, if c 2 = 10, T (n) 10n 2. T (n) = O(n 2 ). Is this the best possible upper bound? Data and Algorithm Analysis Spring / 382 Now, reshape the recurrence so that T is defined for all values of n. T (n) 2T ( n/2 ) + 5n 2 n 2 For arbitrary n, let 2 k 1 < n 2 k. We have already shown that T (2 k ) 10(2 k ) 2. T (n) T (2 k ) 10(2 k ) 2 = 10(2 k /n) 2 n 2 10(2) 2 n 2 40n 2 Hence, T (n) = O(n 2 ) for all values of n. Typically, the bound for powers of two generalizes to all n. Data and Algorithm Analysis Spring / 382 Now, reshape the recurrence so that T is defined for all values of n. T (n) 2T ( n/2 ) + 5n 2 n 2 For arbitrary n, let 2 k 1 < n 2 k. We have already shown that T (2 k ) 10(2 k ) 2. T (n) T (2 k ) 10(2 k ) 2 = 10(2 k /n) 2 n 2 10(2) 2 n 2 40n 2 Hence, T (n) = O(n 2 ) for all values of n. Typically, the bound for powers of two generalizes to all n.

3 k 1 Expanding Recurrences (Intro) Expanding Recurrences (Intro) Expanding Recurrences (Intro) Take a look at the simple example slideshows at OpenDSA. no note Take a look at the simple example slideshows at OpenDSA. Data and Algorithm Analysis Spring / 382 Expanding Recurrences Expanding Recurrences Expanding Recurrences Usually, start with equality version of recurrence. Assume n is a power of 2; n = 2 k. Usually, start with equality version of recurrence. Assume n is a power of 2; n = 2 k. Data and Algorithm Analysis Spring / 382 Expanding Recurrences (cont) Expanding Recurrences (cont) Expanding Recurrences (cont) = 2(2T (n/4) + 5(n/2) 2 ) + 5n 2 = 2(2(2T (n/8) + 5(n/4) 2 ) + 5(n/2) 2 ) + 5n 2 = 2 k T (1) + 2 k 1 5(n/2 k 1 ) k 2 5(n/2 k 2 ) (n/2) 2 + 5n 2 k 1 = 7n + 5 n 2 /2 i = 7n + 5n2 1/2 i = 7n + 5n 2 (2 1/2 k 1 ) = 7n + 5n 2 (2 2/n). = 2(2T (n/4) + 5(n/2) 2 ) + 5n 2 = 2(2(2T (n/8) + 5(n/4) 2 ) + 5(n/2) 2 ) + 5n 2 = 2 k T (1) + 2 k 1 5(n/2 k 1 ) k 2 5(n/2 k 2 ) (n/2) 2 + 5n 2 k 1 k 1 = 7n + 5 n 2 /2 i = 7n + 5n 2 1/2 i = 7n + 5n 2 (2 1/2 k 1 ) = 7n + 5n 2 (2 2/n). This it the exact solution for powers of 2. T (n) = Θ(n 2 ). This it the exact solution for powers of 2. T (n) = Θ(n 2 ). Data and Algorithm Analysis Spring / 382 These have the form: T (n) = at (n/b) + cn k T (1) = c... where a, b, c, k are constants. A problem of size n is divided into a subproblems of size n/b, while cn k is the amount of work needed to combine the solutions. These have the form: T (n) = at (n/b) + cn k T (1) = c... where a, b, c, k are constants. A problem of size n is divided into a subproblems of size n/b, while cn k is the amount of work needed to combine the solutions. Data and Algorithm Analysis Spring / 382

4 (cont) Expand the sum; n = b m. T (n) = a(at (n/b 2 ) + c(n/b) k ) + cn k = a m T (1) + a m 1 c(n/b m 1 ) k + + ac(n/b) k + cn k m = ca m (b k /a) i a m = a log b n = n log b a The summation is a geometric series whose sum depends on the ratio r = b k /a. There are 3 cases. Data and Algorithm Analysis Spring / 382 (cont) n = b m m = log b n. Set a = b log b a. Switch order of logs, giving (b log b n ) log b a = n log b a. (cont) Expand the sum; n = b m. T (n) = a(at (n/b 2 ) + c(n/b) k ) + cn k = a m T (1) + a m 1 c(n/b m 1 ) k + + ac(n/b) k + cn k m = ca m (b k /a) i a m = a log b n = n log b a The summation is a geometric series whose sum depends on the ratio r = b k /a. There are 3 cases. D & C Recurrences (cont) D & C Recurrences (cont) D & C Recurrences (cont) (1) r < 1. (2) r = 1. i r < 1/(1 r), a constant. T (n) = Θ(a m ) = Θ(n log b a ). r i = m + 1 = log b n + 1 (1) r < 1. r i < 1/(1 r), a constant. When r = 1, since r = b k /a = 1, we get a = b k. Recall that k = log b a. T (n) = Θ(n log b a log n) = Θ(n k log n) T (n) = Θ(a m ) = Θ(n log b a ). (2) r = 1. r i = m + 1 = log b n + 1 T (n) = Θ(n log b a log n) = Θ(n k log n) Data and Algorithm Analysis Spring / 382 D & C Recurrences (Case 3) D & C Recurrences (Case 3) (3) r > 1. D & C Recurrences (Case 3) So, from T (n) = ca m r i, r i = r m+1 1 r 1 = Θ(r m ) T (n) = Θ(a m r m ) = Θ(a m (b k /a) m ) = Θ(b km ) = Θ(n k ) (3) r > 1. So, from T (n) = ca m r i, r i = r m+1 1 r 1 = Θ(r m ) T (n) = Θ(a m r m ) = Θ(a m (b k /a) m ) = Θ(b km ) = Θ(n k ) Data and Algorithm Analysis Spring / 382 Summary Summary Theorem 3.4: T (n) = Apply the theorem: T (n) = 3T (n/5) + 8n 2. a = 3, b = 5, c = 8, k = 2. b k /a = 25/3. Summary Θ(n log b a ) Θ(n k log n) Θ(n k ) if a > b k if a = b k if a < b k Case (3) holds: T (n) = Θ(n 2 ). Theorem 3.4: Θ(n log b a ) T (n) = Θ(n k log n) Θ(n k ) if a > b k if a = b k if a < b k We simplify by approximating summations. Apply the theorem: T (n) = 3T (n/5) + 8n 2. a = 3, b = 5, c = 8, k = 2. b k /a = 25/3. Case (3) holds: T (n) = Θ(n 2 ). Data and Algorithm Analysis Spring / 382

5 Examples Examples Examples Mergesort: T (n) = 2T (n/2) + n. 2 1 /2 = 1, so T (n) = Θ(n log n). Binary search: T (n) = T (n/2) /1 = 1, so T (n) = Θ(log n). Insertion sort: T (n) = T (n 1) + n. Can t apply the theorem. Sorry! Standard Matrix Multiply (recursively): T (n) = 8T (n/2) + n /8 = 1/2 so T (n) = Θ(n log 2 8 ) = Θ(n 3 ). Mergesort: T (n) = 2T (n/2) + n. 2 1 /2 = 1, so T (n) = Θ(n log n). Binary search: T (n) = T (n/2) /1 = 1, so T (n) = Θ(log n). Insertion sort: T (n) = T (n 1) + n. Can t apply the theorem. Sorry! Standard Matrix Multiply (recursively): T (n) = 8T (n/2) + n /8 = 1/2 so T (n) = Θ(n log 2 8 ) = Θ(n 3 ). [ c11 c 12 c 21 c 22 ] [ ] [ ] a11 a = 12 b11 b 12 a 21 a 22 b 21 b 22 In the straightforward implementation, 2 2 case is: c 11 = a 11 b 11 + a 12 b 21 c 12 = a 11 b 12 + a 12 b 22 c 21 = a 21 b 11 + a 22 b 21 c 22 = a 21 b 12 + a 22 b 22 So the recursion is 8 calls of half size, and the additions take Θ(n 2 ) work. Data and Algorithm Analysis Spring / 382 Searching Searching Searching Assumptions for search problems: Target is well defined. Target is fixed. Search domain is finite. We (can) remember all information gathered during search. We search for a record with a key. Assumptions for search problems: Target is well defined. Target is fixed. Search domain is finite. We (can) remember all information gathered during search. Well defined: We recognize a hit or miss. Fixed: The target doesn t move during the life of the search. We often choose not to remember information. For example, sequential search does not remember the values seen already. We search for a record with a key. Data and Algorithm Analysis Spring / 382 A Search Model (1) Problem: Given: A list L, of n elements A search key X Solve: Identify one element in L which has key value X, if any exist. Model: The key values for elements in L are unique. One comparison determines <, =, >. Comparison is our only way to find ordering information. Every comparison costs the same. A Search Model (1) A Search Model (1) Problem: Given: A list L, of n elements A search key X Solve: Identify one element in L which has key value X, if any exist. Model: The key values for elements in L are unique. One comparison determines <, =, >. Comparison is our only way to find ordering information. Every comparison costs the same. What if the key values are not unique? Probably the cost goes down, not up. This is an assumption for analysis, not for implementation. We would have a slightly different model (though no asymptotic change in cost) if our only comparison test was <. We would have a very different model if our only comparison was = /. A comparison-based model. String data might require comparisons with very different costs. Data and Algorithm Analysis Spring / 382 A Search Model (2) A Search Model (2) A Search Model (2) Goal: Solve the problem using the minimum number of comparisons. Cost model: Number of comparisons. (Implication) Access to every item in L costs the same (array). Is this a reasonable model and goal? Goal: Solve the problem using the minimum number of comparisons. Cost model: Number of comparisons. (Implication) Access to every item in L costs the same (array). Is this a reasonable model and goal? We are assuming that the # of comparisons is proportional to runtime. Might not always share an array (assumption that all accesses are equal). For example, linked lists. Or data on disk, or across network. We assume there is no relationship between value X and its position. Data and Algorithm Analysis Spring / 382

6 Linear Search General algorithm strategy: Reduce the problem. Compare X to the first element. If not done, then solve the problem for n 1 elements. Position linear_search(l, lower, upper, X) { if L[lower] = X then return lower; else if lower = upper then return -1; else return linear_search(l, lower+1, upper, X); } Linear Search Linear Search General algorithm strategy: Reduce the problem. Compare X to the first element. If not done, then solve the problem for n 1 elements. Position linear_search(l, lower, upper, X) { if L[lower] = X then return lower; else if lower = upper then return -1; else return linear_search(l, lower+1, upper, X); } What equation represents the worst case cost? Warning: We are using this simple, familiar algorithm as an illustration of how to do full, formal analysis. This includes some recurrence solving techniques, and attention to lower bounds. Cost given on next slide. What equation represents the worst case cost? Data and Algorithm Analysis Spring / 382 Worst Cost Upper Bound Worst Cost Upper Bound Worst Cost Upper Bound { 1 n = 1 f (n) = f (n 1) + 1 n > 1 Reasonable to guess that f (n) = n. Prove by induction: Basis step: f (1) = 1, so f (n) = n when n = 1. Induction hypothesis: For k < n, f (k) = k. Induction step: From recurrence, = (n 1) + 1 = n { 1 n = 1 f (n) = f (n 1) + 1 n > 1 Reasonable to guess that f (n) = n. Prove by induction: Basis step: f (1) = 1, so f (n) = n when n = 1. Induction hypothesis: For k < n, f (k) = k. Induction step: From recurrence, = (n 1) + 1 = n Thus, the worst case cost for n elements is linear. Induction is great for verifying a hypothesis. Thus, the worst case cost for n elements is linear. Induction is great for verifying a hypothesis. Data and Algorithm Analysis Spring / 382 Approach #2 What if we couldn t guess a solution? Try: Substitute and Guess. Iterate a few steps of the recurrence, and look for a summation. = {f (n 2) + 1} + 1 = {{f (n 3) + 1} + 1} + 1} Approach #2 Replace i with n 1. Alternative: Recognize f (n) = f (1 n + i=2 1. Approach #2 What if we couldn t guess a solution? Try: Substitute and Guess. Iterate a few steps of the recurrence, and look for a summation. = {f (n 2) + 1} + 1 = {{f (n 3) + 1} + 1} + 1} Now what? Guess f (n) = f (n i) + i. When do we stop? When we reach a value for f that we know. f (n) = f (n (n 1)) + n 1 = f (1) + n 1 = n Now, go back and test the guess using induction. Now what? Guess f (n) = f (n i) + i. When do we stop? When we reach a value for f that we know. f (n) = f (n (n 1)) + n 1 = f (1) + n 1 = n Now, go back and test the guess using induction. Data and Algorithm Analysis Spring / 382 Approach #3 Guess and Test: Guess the form of the solution, then solve the resulting equations. Guess: f (n) is linear. f (n) = rn + s for some r, s. What do we know? f (1) = r 1 + s = r + s = 1. f (n) = r n + s = r (n 1) + s + 1. Solving these two simultaneous equations, r = 1, s = 0. Final form of guess: f (n) = n. Now, prove using induction. Data and Algorithm Analysis Spring / 382 Approach #3 Often, f (0) is easier. Or maybe f (2). Approach #3 Guess and Test: Guess the form of the solution, then solve the resulting equations. Guess: f (n) is linear. f (n) = rn + s for some r, s. What do we know? f (1) = r 1 + s = r + s = 1. f (n) = r n + s = r (n 1) + s + 1. Solving these two simultaneous equations, r = 1, s = 0. Final form of guess: f (n) = n. Now, prove using induction. By definition,, so r n = r (n 1) + 1. So rn + s = rn r + s + 1. s = s r + 1 r 1 = 0 Since. Why is this a guess and not a proof? Because all we did is show that our model passes through two points that the real curve also passes through. If the curve really is linear, 2 points is all that we need. But, we need to prove that it is linear.

7 Lower Bound on the Problem (1) comparing X with L[n]. We can feed the algorithm an input with X in position n. Lower Bound on the Problem (1) Lower Bound on the Problem (1) comparing X with L[n]. We can feed the algorithm an input with X in position n. Be careful about assumptions on how an algorithm might (must) behave. After all, where do new, clever algorithms come from? From different behavior than was previously assumed! Data and Algorithm Analysis Spring / 382 Fixing the Proof (1) Fixing the Proof (1) Fixing the Proof (1) Error #1: An algorithm need not consistently skip position n. Itcould, for example, work from right to left. Fix: Generalize to skip some other position i. Error #1: An algorithm need not consistently skip position n. Itcould, for example, work from right to left. Fix: Generalize to skip some other position i. Data and Algorithm Analysis Spring / 382 Lower Bound on the Problem (2) Lower Bound on the Problem (2) Lower Bound on the Problem (2) comparing X with L[i] for some value i. We can feed the algorithm an input with X in position i. comparing X with L[i] for some value i. We can feed the algorithm an input with X in position i. Data and Algorithm Analysis Spring / 382

/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17

/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17 601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Asymptotic Analysis, recurrences Date: 9/7/17 2.1 Notes Homework 1 will be released today, and is due a week from today by the beginning

More information

CS 577 Introduction to Algorithms: Strassen s Algorithm and the Master Theorem

CS 577 Introduction to Algorithms: Strassen s Algorithm and the Master Theorem CS 577 Introduction to Algorithms: Jin-Yi Cai University of Wisconsin Madison In the last class, we described InsertionSort and showed that its worst-case running time is Θ(n 2 ). Check Figure 2.2 for

More information

COMP Analysis of Algorithms & Data Structures

COMP Analysis of Algorithms & Data Structures COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 10, 2018 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. 1 /

More information

COMP Analysis of Algorithms & Data Structures

COMP Analysis of Algorithms & Data Structures COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Lecture 4 - Jan. 14, 2019 CLRS 1.1, 1.2, 2.2, 3.1, 4.3, 4.5 University of Manitoba Picture is from the cover of the textbook CLRS. COMP

More information

b + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d

b + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d CS161, Lecture 4 Median, Selection, and the Substitution Method Scribe: Albert Chen and Juliana Cook (2015), Sam Kim (2016), Gregory Valiant (2017) Date: January 23, 2017 1 Introduction Last lecture, we

More information

Lecture 3. Big-O notation, more recurrences!!

Lecture 3. Big-O notation, more recurrences!! Lecture 3 Big-O notation, more recurrences!! Announcements! HW1 is posted! (Due Friday) See Piazza for a list of HW clarifications First recitation section was this morning, there s another tomorrow (same

More information

Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort

Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Xi Chen Columbia University We continue with two more asymptotic notation: o( ) and ω( ). Let f (n) and g(n) are functions that map

More information

Asymptotic Analysis and Recurrences

Asymptotic Analysis and Recurrences Appendix A Asymptotic Analysis and Recurrences A.1 Overview We discuss the notion of asymptotic analysis and introduce O, Ω, Θ, and o notation. We then turn to the topic of recurrences, discussing several

More information

CS 310 Advanced Data Structures and Algorithms

CS 310 Advanced Data Structures and Algorithms CS 310 Advanced Data Structures and Algorithms Runtime Analysis May 31, 2017 Tong Wang UMass Boston CS 310 May 31, 2017 1 / 37 Topics Weiss chapter 5 What is algorithm analysis Big O, big, big notations

More information

CIS 121. Analysis of Algorithms & Computational Complexity. Slides based on materials provided by Mary Wootters (Stanford University)

CIS 121. Analysis of Algorithms & Computational Complexity. Slides based on materials provided by Mary Wootters (Stanford University) CIS 121 Analysis of Algorithms & Computational Complexity Slides based on materials provided by Mary Wootters (Stanford University) Today Sorting: InsertionSort vs MergeSort Analyzing the correctness of

More information

MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3

MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3 MA008 p.1/37 MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 3 Dr. Markus Hagenbuchner markus@uow.edu.au. MA008 p.2/37 Exercise 1 (from LN 2) Asymptotic Notation When constants appear in exponents

More information

Mergesort and Recurrences (CLRS 2.3, 4.4)

Mergesort and Recurrences (CLRS 2.3, 4.4) Mergesort and Recurrences (CLRS 2.3, 4.4) We saw a couple of O(n 2 ) algorithms for sorting. Today we ll see a different approach that runs in O(n lg n) and uses one of the most powerful techniques for

More information

CS 4104 Data and Algorithm Analysis. CS4014: What You Need to Already Know. CS4104: What We Will Do. Introduction to Problem Solving (1)

CS 4104 Data and Algorithm Analysis. CS4014: What You Need to Already Know. CS4104: What We Will Do. Introduction to Problem Solving (1) Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 010,017 by Clifford A. Shaffer Data and Algorithm Analysis Title page Data and Algorithm Analysis Clifford A. Shaffer Spring

More information

CS173 Running Time and Big-O. Tandy Warnow

CS173 Running Time and Big-O. Tandy Warnow CS173 Running Time and Big-O Tandy Warnow CS 173 Running Times and Big-O analysis Tandy Warnow Today s material We will cover: Running time analysis Review of running time analysis of Bubblesort Review

More information

CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms

CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms CS 4407 Algorithms Lecture 2: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions

More information

Computational Complexity - Pseudocode and Recursions

Computational Complexity - Pseudocode and Recursions Computational Complexity - Pseudocode and Recursions Nicholas Mainardi 1 Dipartimento di Elettronica e Informazione Politecnico di Milano nicholas.mainardi@polimi.it June 6, 2018 1 Partly Based on Alessandro

More information

CS 2110: INDUCTION DISCUSSION TOPICS

CS 2110: INDUCTION DISCUSSION TOPICS CS 110: INDUCTION DISCUSSION TOPICS The following ideas are suggestions for how to handle your discussion classes. You can do as much or as little of this as you want. You can either present at the board,

More information

The maximum-subarray problem. Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm:

The maximum-subarray problem. Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm: The maximum-subarray problem Given an array of integers, find a contiguous subarray with the maximum sum. Very naïve algorithm: Brute force algorithm: At best, θ(n 2 ) time complexity 129 Can we do divide

More information

5 + 9(10) + 3(100) + 0(1000) + 2(10000) =

5 + 9(10) + 3(100) + 0(1000) + 2(10000) = Chapter 5 Analyzing Algorithms So far we have been proving statements about databases, mathematics and arithmetic, or sequences of numbers. Though these types of statements are common in computer science,

More information

Computational Complexity. This lecture. Notes. Lecture 02 - Basic Complexity Analysis. Tom Kelsey & Susmit Sarkar. Notes

Computational Complexity. This lecture. Notes. Lecture 02 - Basic Complexity Analysis. Tom Kelsey & Susmit Sarkar. Notes Computational Complexity Lecture 02 - Basic Complexity Analysis Tom Kelsey & Susmit Sarkar School of Computer Science University of St Andrews http://www.cs.st-andrews.ac.uk/~tom/ twk@st-andrews.ac.uk

More information

Solving recurrences. Frequently showing up when analysing divide&conquer algorithms or, more generally, recursive algorithms.

Solving recurrences. Frequently showing up when analysing divide&conquer algorithms or, more generally, recursive algorithms. Solving recurrences Frequently showing up when analysing divide&conquer algorithms or, more generally, recursive algorithms Example: Merge-Sort(A, p, r) 1: if p < r then 2: q (p + r)/2 3: Merge-Sort(A,

More information

Analysis of Algorithms - Using Asymptotic Bounds -

Analysis of Algorithms - Using Asymptotic Bounds - Analysis of Algorithms - Using Asymptotic Bounds - Andreas Ermedahl MRTC (Mälardalens Real-Time Research Center) andreas.ermedahl@mdh.se Autumn 004 Rehersal: Asymptotic bounds Gives running time bounds

More information

Methods for solving recurrences

Methods for solving recurrences Methods for solving recurrences Analyzing the complexity of mergesort The merge function Consider the following implementation: 1 int merge ( int v1, int n1, int v, int n ) { 3 int r = malloc ( ( n1+n

More information

A design paradigm. Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/ EECS 3101

A design paradigm. Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/ EECS 3101 A design paradigm Divide and conquer: (When) does decomposing a problem into smaller parts help? 09/09/17 112 Multiplying complex numbers (from Jeff Edmonds slides) INPUT: Two pairs of integers, (a,b),

More information

An analogy from Calculus: limits

An analogy from Calculus: limits COMP 250 Fall 2018 35 - big O Nov. 30, 2018 We have seen several algorithms in the course, and we have loosely characterized their runtimes in terms of the size n of the input. We say that the algorithm

More information

Data Structures and Algorithms Chapter 3

Data Structures and Algorithms Chapter 3 Data Structures and Algorithms Chapter 3 1. Divide and conquer 2. Merge sort, repeated substitutions 3. Tiling 4. Recurrences Recurrences Running times of algorithms with recursive calls can be described

More information

What we have learned What is algorithm Why study algorithm The time and space efficiency of algorithm The analysis framework of time efficiency Asympt

What we have learned What is algorithm Why study algorithm The time and space efficiency of algorithm The analysis framework of time efficiency Asympt Lecture 3 The Analysis of Recursive Algorithm Efficiency What we have learned What is algorithm Why study algorithm The time and space efficiency of algorithm The analysis framework of time efficiency

More information

Divide-and-conquer: Order Statistics. Curs: Fall 2017

Divide-and-conquer: Order Statistics. Curs: Fall 2017 Divide-and-conquer: Order Statistics Curs: Fall 2017 The divide-and-conquer strategy. 1. Break the problem into smaller subproblems, 2. recursively solve each problem, 3. appropriately combine their answers.

More information

data structures and algorithms lecture 2

data structures and algorithms lecture 2 data structures and algorithms 2018 09 06 lecture 2 recall: insertion sort Algorithm insertionsort(a, n): for j := 2 to n do key := A[j] i := j 1 while i 1 and A[i] > key do A[i + 1] := A[i] i := i 1 A[i

More information

Data structures Exercise 1 solution. Question 1. Let s start by writing all the functions in big O notation:

Data structures Exercise 1 solution. Question 1. Let s start by writing all the functions in big O notation: Data structures Exercise 1 solution Question 1 Let s start by writing all the functions in big O notation: f 1 (n) = 2017 = O(1), f 2 (n) = 2 log 2 n = O(n 2 ), f 3 (n) = 2 n = O(2 n ), f 4 (n) = 1 = O

More information

CS 4407 Algorithms Lecture 3: Iterative and Divide and Conquer Algorithms

CS 4407 Algorithms Lecture 3: Iterative and Divide and Conquer Algorithms CS 4407 Algorithms Lecture 3: Iterative and Divide and Conquer Algorithms Prof. Gregory Provan Department of Computer Science University College Cork 1 Lecture Outline CS 4407, Algorithms Growth Functions

More information

Algorithms. Quicksort. Slide credit: David Luebke (Virginia)

Algorithms. Quicksort. Slide credit: David Luebke (Virginia) 1 Algorithms Quicksort Slide credit: David Luebke (Virginia) Sorting revisited We have seen algorithms for sorting: INSERTION-SORT, MERGESORT More generally: given a sequence of items Each item has a characteristic

More information

Computational Complexity

Computational Complexity Computational Complexity S. V. N. Vishwanathan, Pinar Yanardag January 8, 016 1 Computational Complexity: What, Why, and How? Intuitively an algorithm is a well defined computational procedure that takes

More information

Quiz 1 Solutions. Problem 2. Asymptotics & Recurrences [20 points] (3 parts)

Quiz 1 Solutions. Problem 2. Asymptotics & Recurrences [20 points] (3 parts) Introduction to Algorithms October 13, 2010 Massachusetts Institute of Technology 6.006 Fall 2010 Professors Konstantinos Daskalakis and Patrick Jaillet Quiz 1 Solutions Quiz 1 Solutions Problem 1. We

More information

When we use asymptotic notation within an expression, the asymptotic notation is shorthand for an unspecified function satisfying the relation:

When we use asymptotic notation within an expression, the asymptotic notation is shorthand for an unspecified function satisfying the relation: CS 124 Section #1 Big-Oh, the Master Theorem, and MergeSort 1/29/2018 1 Big-Oh Notation 1.1 Definition Big-Oh notation is a way to describe the rate of growth of functions. In CS, we use it to describe

More information

Taking Stock. IE170: Algorithms in Systems Engineering: Lecture 3. Θ Notation. Comparing Algorithms

Taking Stock. IE170: Algorithms in Systems Engineering: Lecture 3. Θ Notation. Comparing Algorithms Taking Stock IE170: Algorithms in Systems Engineering: Lecture 3 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University January 19, 2007 Last Time Lots of funky math Playing

More information

Big O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013

Big O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013 /4/3 Administrative Big O David Kauchak cs3 Spring 3 l Assignment : how d it go? l Assignment : out soon l CLRS code? l Videos Insertion-sort Insertion-sort Does it terminate? /4/3 Insertion-sort Loop

More information

Divide and Conquer CPE 349. Theresa Migler-VonDollen

Divide and Conquer CPE 349. Theresa Migler-VonDollen Divide and Conquer CPE 349 Theresa Migler-VonDollen Divide and Conquer Divide and Conquer is a strategy that solves a problem by: 1 Breaking the problem into subproblems that are themselves smaller instances

More information

When we use asymptotic notation within an expression, the asymptotic notation is shorthand for an unspecified function satisfying the relation:

When we use asymptotic notation within an expression, the asymptotic notation is shorthand for an unspecified function satisfying the relation: CS 124 Section #1 Big-Oh, the Master Theorem, and MergeSort 1/29/2018 1 Big-Oh Notation 1.1 Definition Big-Oh notation is a way to describe the rate of growth of functions. In CS, we use it to describe

More information

Recurrences COMP 215

Recurrences COMP 215 Recurrences COMP 215 Analysis of Iterative Algorithms //return the location of the item matching x, or 0 if //no such item is found. index SequentialSearch(keytype[] S, in, keytype x) { index location

More information

CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2019

CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2019 CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis Ruth Anderson Winter 2019 Today Algorithm Analysis What do we care about? How to compare two algorithms Analyzing Code Asymptotic Analysis

More information

Algorithms Design & Analysis. Analysis of Algorithm

Algorithms Design & Analysis. Analysis of Algorithm Algorithms Design & Analysis Analysis of Algorithm Review Internship Stable Matching Algorithm 2 Outline Time complexity Computation model Asymptotic notions Recurrence Master theorem 3 The problem of

More information

CSC236 Week 4. Larry Zhang

CSC236 Week 4. Larry Zhang CSC236 Week 4 Larry Zhang 1 Announcements PS2 due on Friday This week s tutorial: Exercises with big-oh PS1 feedback People generally did well Writing style need to be improved. This time the TAs are lenient,

More information

Divide and Conquer. Arash Rafiey. 27 October, 2016

Divide and Conquer. Arash Rafiey. 27 October, 2016 27 October, 2016 Divide the problem into a number of subproblems Divide the problem into a number of subproblems Conquer the subproblems by solving them recursively or if they are small, there must be

More information

Big , and Definition Definition

Big , and Definition Definition Big O, Ω, and Θ Big-O gives us only a one-way comparison; if f is O(g) then g eventually is bigger than f from that point on, but in fact f could be very small in comparison. Example; 3n is O(2 2n ). We

More information

CS 161 Summer 2009 Homework #2 Sample Solutions

CS 161 Summer 2009 Homework #2 Sample Solutions CS 161 Summer 2009 Homework #2 Sample Solutions Regrade Policy: If you believe an error has been made in the grading of your homework, you may resubmit it for a regrade. If the error consists of more than

More information

Design and Analysis of Algorithms

Design and Analysis of Algorithms CSE 101, Winter 2018 Design and Analysis of Algorithms Lecture 5: Divide and Conquer (Part 2) Class URL: http://vlsicad.ucsd.edu/courses/cse101-w18/ A Lower Bound on Convex Hull Lecture 4 Task: sort the

More information

Analysis of Algorithm Efficiency. Dr. Yingwu Zhu

Analysis of Algorithm Efficiency. Dr. Yingwu Zhu Analysis of Algorithm Efficiency Dr. Yingwu Zhu Measure Algorithm Efficiency Time efficiency How fast the algorithm runs; amount of time required to accomplish the task Our focus! Space efficiency Amount

More information

UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis

UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis UCSD CSE 21, Spring 2014 [Section B00] Mathematics for Algorithm and System Analysis Lecture 14 Class URL: http://vlsicad.ucsd.edu/courses/cse21-s14/ Lecture 14 Notes Goals for this week Big-O complexity

More information

Algorithms, Design and Analysis. Order of growth. Table 2.1. Big-oh. Asymptotic growth rate. Types of formulas for basic operation count

Algorithms, Design and Analysis. Order of growth. Table 2.1. Big-oh. Asymptotic growth rate. Types of formulas for basic operation count Types of formulas for basic operation count Exact formula e.g., C(n) = n(n-1)/2 Algorithms, Design and Analysis Big-Oh analysis, Brute Force, Divide and conquer intro Formula indicating order of growth

More information

Algorithm Analysis Recurrence Relation. Chung-Ang University, Jaesung Lee

Algorithm Analysis Recurrence Relation. Chung-Ang University, Jaesung Lee Algorithm Analysis Recurrence Relation Chung-Ang University, Jaesung Lee Recursion 2 Recursion 3 Recursion in Real-world Fibonacci sequence = + Initial conditions: = 0 and = 1. = + = + = + 0, 1, 1, 2,

More information

Grade 11/12 Math Circles Fall Nov. 5 Recurrences, Part 2

Grade 11/12 Math Circles Fall Nov. 5 Recurrences, Part 2 1 Faculty of Mathematics Waterloo, Ontario Centre for Education in Mathematics and Computing Grade 11/12 Math Circles Fall 2014 - Nov. 5 Recurrences, Part 2 Running time of algorithms In computer science,

More information

A SUMMARY OF RECURSION SOLVING TECHNIQUES

A SUMMARY OF RECURSION SOLVING TECHNIQUES A SUMMARY OF RECURSION SOLVING TECHNIQUES KIMMO ERIKSSON, KTH These notes are meant to be a complement to the material on recursion solving techniques in the textbook Discrete Mathematics by Biggs. In

More information

Data Structures and Algorithms Chapter 3

Data Structures and Algorithms Chapter 3 1 Data Structures and Algorithms Chapter 3 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples

More information

CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2018

CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2018 CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis Ruth Anderson Winter 2018 Today Algorithm Analysis What do we care about? How to compare two algorithms Analyzing Code Asymptotic Analysis

More information

CS 5114: Theory of Algorithms. Tractable Problems. Tractable Problems (cont) Decision Problems. Clifford A. Shaffer. Spring 2014

CS 5114: Theory of Algorithms. Tractable Problems. Tractable Problems (cont) Decision Problems. Clifford A. Shaffer. Spring 2014 Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2014 by Clifford A. Shaffer : Theory of Algorithms Title page : Theory of Algorithms Clifford A. Shaffer Spring 2014 Clifford

More information

Algorithm efficiency analysis

Algorithm efficiency analysis Algorithm efficiency analysis Mădălina Răschip, Cristian Gaţu Faculty of Computer Science Alexandru Ioan Cuza University of Iaşi, Romania DS 2017/2018 Content Algorithm efficiency analysis Recursive function

More information

CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2018

CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis. Ruth Anderson Winter 2018 CSE332: Data Structures & Parallelism Lecture 2: Algorithm Analysis Ruth Anderson Winter 2018 Today Algorithm Analysis What do we care about? How to compare two algorithms Analyzing Code Asymptotic Analysis

More information

Topic 17. Analysis of Algorithms

Topic 17. Analysis of Algorithms Topic 17 Analysis of Algorithms Analysis of Algorithms- Review Efficiency of an algorithm can be measured in terms of : Time complexity: a measure of the amount of time required to execute an algorithm

More information

Growth of Functions (CLRS 2.3,3)

Growth of Functions (CLRS 2.3,3) Growth of Functions (CLRS 2.3,3) 1 Review Last time we discussed running time of algorithms and introduced the RAM model of computation. Best-case running time: the shortest running time for any input

More information

In-Class Soln 1. CS 361, Lecture 4. Today s Outline. In-Class Soln 2

In-Class Soln 1. CS 361, Lecture 4. Today s Outline. In-Class Soln 2 In-Class Soln 1 Let f(n) be an always positive function and let g(n) = f(n) log n. Show that f(n) = o(g(n)) CS 361, Lecture 4 Jared Saia University of New Mexico For any positive constant c, we want to

More information

CS2223 Algorithms D Term 2009 Exam 3 Solutions

CS2223 Algorithms D Term 2009 Exam 3 Solutions CS2223 Algorithms D Term 2009 Exam 3 Solutions May 4, 2009 By Prof. Carolina Ruiz Dept. of Computer Science WPI PROBLEM 1: Asymptoptic Growth Rates (10 points) Let A and B be two algorithms with runtimes

More information

Solving Recurrences. 1. Express the running time (or use of some other resource) as a recurrence.

Solving Recurrences. 1. Express the running time (or use of some other resource) as a recurrence. Solving Recurrences Recurrences and Recursive Code Many (perhaps most) recursive algorithms fall into one of two categories: tail recursion and divide-andconquer recursion. We would like to develop some

More information

Divide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,

Divide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30, Divide and Conquer CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Merging sorted lists: WHAT Given two sorted lists a 1 a 2 a 3 a k b 1 b 2 b 3 b

More information

Advanced Counting Techniques. Chapter 8

Advanced Counting Techniques. Chapter 8 Advanced Counting Techniques Chapter 8 Chapter Summary Applications of Recurrence Relations Solving Linear Recurrence Relations Homogeneous Recurrence Relations Nonhomogeneous Recurrence Relations Divide-and-Conquer

More information

Review Of Topics. Review: Induction

Review Of Topics. Review: Induction Review Of Topics Asymptotic notation Solving recurrences Sorting algorithms Insertion sort Merge sort Heap sort Quick sort Counting sort Radix sort Medians/order statistics Randomized algorithm Worst-case

More information

1 Substitution method

1 Substitution method Recurrence Relations we have discussed asymptotic analysis of algorithms and various properties associated with asymptotic notation. As many algorithms are recursive in nature, it is natural to analyze

More information

i=1 i B[i] B[i] + A[i, j]; c n for j n downto i + 1 do c n i=1 (n i) C[i] C[i] + A[i, j]; c n

i=1 i B[i] B[i] + A[i, j]; c n for j n downto i + 1 do c n i=1 (n i) C[i] C[i] + A[i, j]; c n Fundamental Algorithms Homework #1 Set on June 25, 2009 Due on July 2, 2009 Problem 1. [15 pts] Analyze the worst-case time complexity of the following algorithms,and give tight bounds using the Theta

More information

COL106: Data Structures and Algorithms (IIT Delhi, Semester-II )

COL106: Data Structures and Algorithms (IIT Delhi, Semester-II ) 1 Solve the following recurrence relations giving a Θ bound for each of the cases 1 : (a) T (n) = 2T (n/3) + 1; T (1) = 1 (Assume n is a power of 3) (b) T (n) = 5T (n/4) + n; T (1) = 1 (Assume n is a power

More information

Math 3361-Modern Algebra Lecture 08 9/26/ Cardinality

Math 3361-Modern Algebra Lecture 08 9/26/ Cardinality Math 336-Modern Algebra Lecture 08 9/26/4. Cardinality I started talking about cardinality last time, and you did some stuff with it in the Homework, so let s continue. I said that two sets have the same

More information

Notes for Recitation 14

Notes for Recitation 14 6.04/18.06J Mathematics for Computer Science October 4, 006 Tom Leighton and Marten van Dijk Notes for Recitation 14 1 The Akra-Bazzi Theorem Theorem 1 (Akra-Bazzi, strong form). Suppose that: is defined

More information

Divide and Conquer. Recurrence Relations

Divide and Conquer. Recurrence Relations Divide and Conquer Recurrence Relations Divide-and-Conquer Strategy: Break up problem into parts. Solve each part recursively. Combine solutions to sub-problems into overall solution. 2 MergeSort Mergesort.

More information

Introduction to Divide and Conquer

Introduction to Divide and Conquer Introduction to Divide and Conquer Sorting with O(n log n) comparisons and integer multiplication faster than O(n 2 ) Periklis A. Papakonstantinou York University Consider a problem that admits a straightforward

More information

Analysis of Algorithms [Reading: CLRS 2.2, 3] Laura Toma, csci2200, Bowdoin College

Analysis of Algorithms [Reading: CLRS 2.2, 3] Laura Toma, csci2200, Bowdoin College Analysis of Algorithms [Reading: CLRS 2.2, 3] Laura Toma, csci2200, Bowdoin College Why analysis? We want to predict how the algorithm will behave (e.g. running time) on arbitrary inputs, and how it will

More information

Solving Recurrences. 1. Express the running time (or use of some other resource) as a recurrence.

Solving Recurrences. 1. Express the running time (or use of some other resource) as a recurrence. 1 Recurrences and Recursive Code Solving Recurrences Many (perhaps most) recursive algorithms fall into one of two categories: tail recursion and divide-andconquer recursion. We would like to develop some

More information

CS483 Design and Analysis of Algorithms

CS483 Design and Analysis of Algorithms CS483 Design and Analysis of Algorithms Chapter 2 Divide and Conquer Algorithms Instructor: Fei Li lifei@cs.gmu.edu with subject: CS483 Office hours: Room 5326, Engineering Building, Thursday 4:30pm -

More information

Omega notation. Transitivity etc.

Omega notation. Transitivity etc. Omega notation Big-Omega: Lecture 2, Sept. 25, 2014 f () n (()) g n const cn, s.t. n n : cg() n f () n Small-omega: 0 0 0 f () n (()) g n const c, n s.t. n n : cg() n f () n 0 0 0 Intuition (works most

More information

Recurrence Relations

Recurrence Relations Recurrence Relations Analysis Tools S.V. N. (vishy) Vishwanathan University of California, Santa Cruz vishy@ucsc.edu January 15, 2016 S.V. N. Vishwanathan (UCSC) CMPS101 1 / 29 Recurrences Outline 1 Recurrences

More information

Lecture 2: Asymptotic Notation CSCI Algorithms I

Lecture 2: Asymptotic Notation CSCI Algorithms I Lecture 2: Asymptotic Notation CSCI 700 - Algorithms I Andrew Rosenberg September 2, 2010 Last Time Review Insertion Sort Analysis of Runtime Proof of Correctness Today Asymptotic Notation Its use in analyzing

More information

Divide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch]

Divide and Conquer. Andreas Klappenecker. [based on slides by Prof. Welch] Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch] Divide and Conquer Paradigm An important general technique for designing algorithms: divide problem into subproblems recursively

More information

Asymptotic Analysis 1

Asymptotic Analysis 1 Asymptotic Analysis 1 Last week, we discussed how to present algorithms using pseudocode. For example, we looked at an algorithm for singing the annoying song 99 Bottles of Beer on the Wall for arbitrary

More information

Data Structures and Algorithms CMPSC 465

Data Structures and Algorithms CMPSC 465 Data Structures and Algorithms CMPSC 465 LECTURE 9 Solving recurrences Substitution method Adam Smith S. Raskhodnikova and A. Smith; based on slides by E. Demaine and C. Leiserson Review question Draw

More information

Recurrence Relations

Recurrence Relations Recurrence Relations Winter 2017 Recurrence Relations Recurrence Relations A recurrence relation for the sequence {a n } is an equation that expresses a n in terms of one or more of the previous terms

More information

Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018

Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 CS17 Integrated Introduction to Computer Science Klein Contents Lecture 17: Trees and Merge Sort 10:00 AM, Oct 15, 2018 1 Tree definitions 1 2 Analysis of mergesort using a binary tree 1 3 Analysis of

More information

Divide and Conquer Algorithms. CSE 101: Design and Analysis of Algorithms Lecture 14

Divide and Conquer Algorithms. CSE 101: Design and Analysis of Algorithms Lecture 14 Divide and Conquer Algorithms CSE 101: Design and Analysis of Algorithms Lecture 14 CSE 101: Design and analysis of algorithms Divide and conquer algorithms Reading: Sections 2.3 and 2.4 Homework 6 will

More information

Problem Set 1. CSE 373 Spring Out: February 9, 2016

Problem Set 1. CSE 373 Spring Out: February 9, 2016 Problem Set 1 CSE 373 Spring 2016 Out: February 9, 2016 1 Big-O Notation Prove each of the following using the definition of big-o notation (find constants c and n 0 such that f(n) c g(n) for n > n o.

More information

Lecture 3: Big-O and Big-Θ

Lecture 3: Big-O and Big-Θ Lecture 3: Big-O and Big-Θ COSC4: Algorithms and Data Structures Brendan McCane Department of Computer Science, University of Otago Landmark functions We saw that the amount of work done by Insertion Sort,

More information

ITEC2620 Introduction to Data Structures

ITEC2620 Introduction to Data Structures ITEC2620 Introduction to Data Structures Lecture 6a Complexity Analysis Recursive Algorithms Complexity Analysis Determine how the processing time of an algorithm grows with input size What if the algorithm

More information

CS Analysis of Recursive Algorithms and Brute Force

CS Analysis of Recursive Algorithms and Brute Force CS483-05 Analysis of Recursive Algorithms and Brute Force Instructor: Fei Li Room 443 ST II Office hours: Tue. & Thur. 4:30pm - 5:30pm or by appointments lifei@cs.gmu.edu with subject: CS483 http://www.cs.gmu.edu/

More information

V. Adamchik 1. Recurrences. Victor Adamchik Fall of 2005

V. Adamchik 1. Recurrences. Victor Adamchik Fall of 2005 V. Adamchi Recurrences Victor Adamchi Fall of 00 Plan Multiple roots. More on multiple roots. Inhomogeneous equations 3. Divide-and-conquer recurrences In the previous lecture we have showed that if the

More information

Module 1: Analyzing the Efficiency of Algorithms

Module 1: Analyzing the Efficiency of Algorithms Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu What is an Algorithm?

More information

Outline. 1 Introduction. Merging and MergeSort. 3 Analysis. 4 Reference

Outline. 1 Introduction. Merging and MergeSort. 3 Analysis. 4 Reference Outline Computer Science 331 Sort Mike Jacobson Department of Computer Science University of Calgary Lecture #25 1 Introduction 2 Merging and 3 4 Reference Mike Jacobson (University of Calgary) Computer

More information

Lecture 2. More Algorithm Analysis, Math and MCSS By: Sarah Buchanan

Lecture 2. More Algorithm Analysis, Math and MCSS By: Sarah Buchanan Lecture 2 More Algorithm Analysis, Math and MCSS By: Sarah Buchanan Announcements Assignment #1 is posted online It is directly related to MCSS which we will be talking about today or Monday. There are

More information

Practical Session #3 - Recursions

Practical Session #3 - Recursions Practical Session #3 - Recursions Substitution method Guess the form of the solution and prove it by induction Iteration Method Convert the recurrence into a summation and solve it Tightly bound a recurrence

More information

1 Closest Pair of Points on the Plane

1 Closest Pair of Points on the Plane CS 31: Algorithms (Spring 2019): Lecture 5 Date: 4th April, 2019 Topic: Divide and Conquer 3: Closest Pair of Points on a Plane Disclaimer: These notes have not gone through scrutiny and in all probability

More information

University of New Mexico Department of Computer Science. Midterm Examination. CS 361 Data Structures and Algorithms Spring, 2003

University of New Mexico Department of Computer Science. Midterm Examination. CS 361 Data Structures and Algorithms Spring, 2003 University of New Mexico Department of Computer Science Midterm Examination CS 361 Data Structures and Algorithms Spring, 2003 Name: Email: Print your name and email, neatly in the space provided above;

More information

Section Summary. Sequences. Recurrence Relations. Summations. Examples: Geometric Progression, Arithmetic Progression. Example: Fibonacci Sequence

Section Summary. Sequences. Recurrence Relations. Summations. Examples: Geometric Progression, Arithmetic Progression. Example: Fibonacci Sequence Section 2.4 Section Summary Sequences. Examples: Geometric Progression, Arithmetic Progression Recurrence Relations Example: Fibonacci Sequence Summations Introduction Sequences are ordered lists of elements.

More information

Recursion: Introduction and Correctness

Recursion: Introduction and Correctness Recursion: Introduction and Correctness CSE21 Winter 2017, Day 7 (B00), Day 4-5 (A00) January 25, 2017 http://vlsicad.ucsd.edu/courses/cse21-w17 Today s Plan From last time: intersecting sorted lists and

More information

Lecture 2. Fundamentals of the Analysis of Algorithm Efficiency

Lecture 2. Fundamentals of the Analysis of Algorithm Efficiency Lecture 2 Fundamentals of the Analysis of Algorithm Efficiency 1 Lecture Contents 1. Analysis Framework 2. Asymptotic Notations and Basic Efficiency Classes 3. Mathematical Analysis of Nonrecursive Algorithms

More information

CS 470/570 Divide-and-Conquer. Format of Divide-and-Conquer algorithms: Master Recurrence Theorem (simpler version)

CS 470/570 Divide-and-Conquer. Format of Divide-and-Conquer algorithms: Master Recurrence Theorem (simpler version) CS 470/570 Divide-and-Conquer Format of Divide-and-Conquer algorithms: Divide: Split the array or list into smaller pieces Conquer: Solve the same problem recursively on smaller pieces Combine: Build the

More information