CS 344 Design and Analysis of Algorithms. Tarek El-Gaaly Course website:

Transcription

1 CS 344 Design and Analysis of Algorithms Tarek El-Gaaly Course website:

2 Course Outline Textbook: Algorithms by S. Dasgupta, C.H. Papadimitriou, and U.V. Vazirani ebook Chapters: Prologue Today: growth rates of functions Algorithms with numbers Divide & Conquer Decomposition of graphs Paths in graphs Greedy Algorithms Dynamic Programming NP-complete problems Coping with NP-completeness No LP and Quantum Computing Good ref: Intro to Algorithms, Cormen et al.

3 What makes a good algorithm? Simplicity in implementation (debuggable) Parallelizability Space complexity Time complexity Worst case performance? When input size gets big! Growth rates Recursive vs. Iterative? E.g. Towers of Hanoi

4 Runtime of an algorithm and comparing with others Growth rates give a simple measure of efficiency and allow comparison of algorithms In terms of n (input size) Example: When n gets large Merge-sort: θ(n logn) Insertion Sort: θ(n 2 ) [best case θ(n)] Wikipedia: Sorting Algorithms - Wikipedia: Big O Notation -

5 Asymptotic Analysis Asymptote: in analytical geometry line which a curve approaches at infinity (bound) When things get big what are the limits? Logarithmic Polylogarithmic Sublinear (e.g. n.5 = n Linear Quadratic Exponential

6 Definitions: θ-notation θ-notation If c1, c2 and n0 exist Example: 1 2 n2 3n = θ(n 2 ) Must determine c 1, c 2, n o s.t c 1 n n2 3n c 2 n 2, for all n n o

7 Definitions: O-notation O-notation O(g n ) = {f n : there exists pos. const. c and n o s.t. 0 f n cg n for all n n o } Bounded above

8 Definitions: Ω-notation Ω-notation Ω g n = {f n : there exists pos. const. c and n o s.t. 0 cg n f(n) for all n n o } Asymptotically bounded below Bounded best-case of an algorithm (e.g. insertion sort: Ω(n)) Insertion sort: tightly bound below by Ω(n) and tightly bound above by O n 2

9 Definition: o-notation o-notation o(g n = {f(n): for any pos. const. c > 0, there exists a const. n o > 0 s.t. 0 f n < cg n for all n n o } Asymptotically loosely bound

10 Definition: ω-notation ω-notation Related to Ω as o is to O ω g n = {f n : for any pos. const. c > 0, there exists a const. n o s.t. 0 cg n < f(n) for all n n o Lower loose bound

11 Asymptotic Notation Exact complexity is not worth the effort n + 2 operations is no different from n operations Larger order terms dominate: n 2 + n + 1 O bounded above Ω bounded below θ bounded above and below o loosely bound from above ω loosely bound from below

12 O, θ, Ω Asymptotic Analysis

13 Analogous to Inequalities f = O(g) f g (asymptotic upper bound) f = Ω(g) f g (asymptotic lower bound) f = θ(g) f = g (asymptotic tight bound) f = o(g) f < g (asymptotic loose upper bound) f = ω(g) f > g (asymptotic loose lower bound)

14 Rules of Thumb Multiplicative constants can be omitted 14n 2 n 2 n a dominates n b if a > b e.g. n 2 dominates n Any exponential dominates any polynomial 3 n dominates n 5 (and even 2 n ) Any polynomial dominates any logarithm n dominates logn 4

15 Rules of Thumb

16 Rules of Thumb

17 Rules of Thumb f = Ω g g = O(f) f = θ n f = O g and f = Ω(g) f = O g and g = O f f = θ(g)

18 Limits Lim f g Lim f g = f = ω(g) = 0 f = o(g) Real-life O, θ What happens if you get or 0 0 (undefined values) L Hopital s rule

19 L Hopital s Rule Lim f g is the same as Lim f g Which is the same as Lim f g And so on Intuitively: compare the rate of change of the rate of change etc.

20 Logarithmic Order Binary search (search sorted data) Select middle element of array (i.e median) Compares it against a target value. If the values match it will return success. If lower than target take upper half of data If higher than target take lower half Continue O(log N) Iterative halving of data produces a growth curve that peaks at the beginning and slowly flattens out as size of the data increase e.g. 10 items takes one second to complete, 100 items takes two seconds, and 1000 items will take three seconds. Doubling the size of the input data set has little effect on its growth. This makes algorithms like binary search extremely efficient when dealing with large data

21 Logarithmic Order

22 Logarithmic Rules Intro to Algorithms: Change of base

23 Hierarchy of Functions

24 Hierarchy of Functions

25 Examples Determine if: f = O(g) or f = Ω(g) or both: f = θ(g)? f g Intuitively Rules of Thumb Limits (L Hopital s Rule) Apply log What does nlogn mean intuitively it means the problem space is dividing (by two) recursively i.e. binary search, merge-sort, n 4 + n 2 n 5 n 2 n 2 nlogn 4 log 2 n 3 n 2 n

26 Examples If you can find the constant then it becomes big-oh or big-omega

27 Some Additional Notes n! = o(n n ) n! = ω(2 n ) Log(n!) = θ(nlogn)

28 Summary When comparing functions: Use rule of thumbs Use direct substitution Look at limits Apply log

29 Examples Problem: Remove duplicates from an unsorted array Sort: O(n logn) Quicksort One sweep through the array will find duplicates. Copy the nonduplicates elements to a new array Overall O(nlogn + n) = O(nlogn) Use a hashtable of the elements to check if duplicate Space complexity? Another array is needed Alternative: do it in-place One solution: Go through the array, build a hashtable and mark the duplicates and move nonduplicates back into the hole (still space complexity is double not in place) If elements are limited between 0 and n think about it!