Introduction to Algorithms: Asymptotic Notation

Transcription

1 Introduction to Algorithms: Asymptotic Notation

2 Why Should We Care? (Order of growth) CS Analysis of Algorithms 2

3 Asymptotic Notation O-notation (upper bounds): that 0 f(n) cg(n) for all n n. 0 CS Analysis of Algorithms 3

4 Asymptotic Notation O-notation (upper bounds): that 0 f(n) cg(n) for all n n. 0 CS Analysis of Algorithms 4

5 Asymptotic Notation O-notation (upper bounds): that 0 f(n) cg(n) for all n n. 0 where c = 1, n 0 = 2 CS Analysis of Algorithms 5

6 Asymptotic Notation O-notation (upper bounds): that 0 f(n) cg(n) for all n n. 0 functions, not values CS Analysis of Algorithms where c = 1, n 0 = 2 funny, one-way equality 6

7 Asymptotic Notation O-notation (upper bounds): that 0 f(n) cg(n) for all n n. 0

8 Asymptotic Notation O-notation (upper bounds): that 0 f(n) cg(n) for all n n. 0

9 Set Definition of Ο-notation Ο(g n ) is the set of all functions with a SMALLER or the SAME order of growth as g n. CS Analysis of Algorithms 9

10 Set Definition of Ο-notation Ο(g n ) is the set of all functions with a SMALLER or the SAME order of growth as f(n). EXAMPLE: Ο(n ' ) = 5n ', 10n , logn, Or put another way: 5n ' Ο(n ' ) CS Analysis of Algorithms 10

11 Ω-notation (lower bounds)

12 Asymptotic Notation Ω-notation (lower bounds): that 0 f(n) cg(n) for all n n. 0 CS Analysis of Algorithms 12

13 Asymptotic Notation Ω-notation (lower bounds): that 0 f(n) cg(n) for all n n. 0 CS Analysis of Algorithms 13

14 Asymptotic Notation Ω-notation (lower bounds): that 0 f(n) cg(n) for all n n. 0 where c = 1, n 0 = 16 CS Analysis of Algorithms 14

15 Asymptotic Notation Ω-notation (lower bounds): that 0 f(n) cg(n) for all n n. 0 CS Analysis of Algorithms 15

16 Set Definition of Ω-notation Ω(g n ) is the set of all functions with a LARGER or the SAME order of growth as g n. EXAMPLE: Ω(n - ) = 5n -, 10n ' + 500, n - log n, Or put another way: 5n - Ω(n - ) CS Analysis of Algorithms 16

17 Θ-notation (tight bounds) Combine definitions of Ο and Ω: We write: f n = Θ g n if there exist constants c 1 > 0, c 2 > 0 and n 0 > 0 such that c 1 g n <= f n <= c 2 g(n) CS Analysis of Algorithms 17

18 Θ-notation (tight bounds) Combine definitions of Ο and Ω: We write: f n = Θ g n if there exist constants c 1 > 0, c 2 > 0 and n 0 > 0 such that c 1 g n <= f n <= c 2 g(n) CS Analysis of Algorithms 18

19 Θ-notation (tight bounds) CS Analysis of Algorithms 19

20 Θ-notation (tight bounds) CS Analysis of Algorithms 20

21 Θ-notation (tight bounds) Upper bound CS Analysis of Algorithms 21

22 Θ-notation (tight bounds) Upper bound Lower bound CS Analysis of Algorithms 22

23 Θ-notation (tight bounds) Upper bound Lower bound CS Analysis of Algorithms 23

24 Θ-notation (tight bounds) EXAMPLE: CS Analysis of Algorithms 24

25 O-notation like. o-notation like <. ο-notation We write: f n = ο g n for any constants c > 0, there is a constant n M > 0 such that 0 < f(n) < cg(n) CS Analysis of Algorithms 25

26 O-notation like. o-notation like <. ο-notation We write: f n = ο g n for any constants c > 0, there is a constant n M > 0 such that 0 < f(n) < cg(n) EXAMPLE: 2n 2 = o(n 3 ), n 0 = 2/c CS Analysis of Algorithms 26

27 Set Definition of ο-notation ο(g n ) is the set of all functions with a strictly SMALLER order of growth as g n. CS Analysis of Algorithms 27

28 Set Definition of ο-notation ο(g n ) is the set of all functions with a SMALLER order of growth as g n. EXAMPLE: ο(n ' ) = 10n , log n, Or put another way: 10n ο(n ' ) CS Analysis of Algorithms 28

29 Ω-notation is like. ω-notation is like >. ω-notation We write: f n = ω g n for any constants c > 0, there is a constant n M > 0 such that 0 < cg(n) < f(n) CS Analysis of Algorithms 29

30 Ω-notation is like. ω-notation is like >. ω-notation We write: f n = ω g n for any constants c > 0, there is a constant n M > 0 such that 0 < cg(n) < f(n) EXAMPLE: n = ω(lg n), CS Analysis of Algorithms n 0 = 1+1/c 30

31 Set Definition of ω-notation ω(f(n)) is the set of all functions with a strictly LARGER order of growth as f(n). CS Analysis of Algorithms 31

32 Set Definition of ω-notation ω(f(n)) is the set of all functions with a strictly LARGER order of growth as f(n). EXAMPLE: ω(n - ) = 10n ' + 500, n - logn, Or put another way: 10n ' ω(n - ) CS Analysis of Algorithms 32

33 Properties: Asymptotic Notation Transitivity Reflexivity Symmetry Transposition CS Analysis of Algorithms 33

34 Transitivity Assuming f(n) and g(n) are asymptotically positive: f(n) = Θ(g(n)) and g(n) = Θ(h(n)) implies f(n) = Θ(h(n)) Holds for Ο, Ω, ο, and ω relations as well. CS Analysis of Algorithms 34

35 Reflexivity Assuming f(n) is asymptotically positive: f(n) = Ο(f(n)) and f(n) = Ω(f(n)) and f(n) = Θ(f(n)) DOES NOT hold for ο and ω relations. CS Analysis of Algorithms 35

36 Symmetry Assuming f(n) and g(n) are asymptotically positive: f(n) = Θ(g(n)) iff (if, and only if,) g(n) = Θ(f(n)) CS Analysis of Algorithms 36

37 Transpose Assuming f(n) and g(n) are asymptotically positive: f(n) = Ο(g(n)) iff g(n) = Ω(f(n)) and f(n) = ο(g(n)) iff g(n) = ω(f(n)) CS Analysis of Algorithms 37

38 Analogy with numbers CS Analysis of Algorithms 38

39 Analogy with numbers CS Analysis of Algorithms 39

40 Analogy with numbers CS Analysis of Algorithms 40

41 Useful Property f1(n) = Θ(g1(n)) and f2 n = Θ(g2(n)) implies f1 n + f2 n = Θ(max {g1(n), g2(n)}) Holds for Ο, Ω, ο, and ω relations as well. CS Analysis of Algorithms 41

42 Using Limits to Compare Growth Rates Though using Ο, Ω, ο, and ω indispensable for comparing growth rates of functions in the abstract, when comparing actual functions, convenient to CS Analysis of Algorithms 42

43 Using Limits to Compare Growth Rates lim V X f(n) g(n) = 0 - f(n) has smaller growth rate than g(n) c - f(n) has same growth rate as g(n) - f(n) has larger growth rate than g(n) first two cases f n Ο g n last two cases f n Ω(g n ) second case f n Θ(g n ) CS Analysis of Algorithms 43

44 Growth Rates (Example 1) CS Analysis of Algorithms 44

45 Growth Rates (Example 1) CS Analysis of Algorithms 45

46 Growth Rates (Example 1) CS Analysis of Algorithms 46

47 Growth Rates (Example 2) CS Analysis of Algorithms 47

48 Growth Rates (Example 2) CS Analysis of Algorithms 48

49 Growth Rates (Example 2) L'Hôpital's rule CS Analysis of Algorithms 49

50 Growth Rates (Example 2) L'Hôpital's rule CS Analysis of Algorithms 50

51 Growth Rates (Example 2) Derivatives ath/calculus/differentiation/list_of_derivatives CS Analysis of Algorithms 51

52 Growth Rates (Example 2) CS Analysis of Algorithms 52

53 Growth Rates (Example 3) CS Analysis of Algorithms 53

54 Growth Rates (Example 3) Stirling's approximation CS Analysis of Algorithms 54

55 Growth Rates (Example 3) CS Analysis of Algorithms 55

56 Growth Rates (Example 3) CS Analysis of Algorithms 56

57 Most Common Growth Rates Class Name Examples 1 Constant Only used in best-case efficiencies. log n logarithmic Result of cutting problem size by a constant factor, like Binary Search n linear Algorithms that scan a list of size n, like Sequential, or Linear, Search n log n n-log-n Divide-and-Conquer algorithms, like Merge Sort and Quick Sort n2 quadratic Efficiencies with two embedded loops, Bubble Sort and Insertion Sort n3 cubic Efficiencies with three embedded loops, like many linear algebra algorithms 2n exponential Algorithms that generate all sub-sets of an n-element set n! factorial Algorithms that generate all permutations CS Analysis of an of Algorithms n-element set 57

58 Macro Substitution Convention: A set in a formula represents an anonymous function in the set.

59 Macro substitution Convention: A set in a formula represents an anonymous function in the set. EXAMPLE: f(n) = n 3 + O(n 2 ) means f(n) = n 3 + h(n) for some h(n) O(n 2 ) CS Analysis of Algorithms 59

60 Macro substitution Convention: A set in a formula represents an anonymous function in the set. EXAMPLE: CS Analysis of Algorithms 60