Asymptotic notation. O-notation 2 78 and g(n) = 1 2 n2.then. O-notation 3 Example. Let f (n) =n 3 and g(n) =2n 2.Thenf (n) 6= O(g(n)).

Transcription

1 Asymptotic notation Reading: Cormen et al, Section 3.1 In the analysis of algorithms, it is common to estimate the running time in the asymptotic sense, that is, to estimate the running time for arbitrarily large inputs. The notations O,,, o and! are used to denote various kinds of asymptotic behaviour. These notations are defined for whose domain is the set of natural numbers N = {0, 1,,...}. It is assumed throughout that all f (n) usedwithinthe asymptotic notations are asymptotically non-negative, thatis,f (n) is nonnegative whenever n is su ciently large. Example. Let f (n) = 7 n f (n) =O(g(n)). O-notation 78 and g(n) = 1 n.then To prove this, we have to find positive constants c and n 0 such that 0 apple f (n) apple cg(n), that is, 0 apple 7 n 78 apple c 1 n, 0 apple 7 78 n apple c, Since 7 78 can be made arbitrarily small by choosing large n enough values of n, any choice of c > 0 works for all su ciently large n (e.g., c =1andn 0 = 30 work). COMP3600/6466 Algorithms 018 Lecture 1 1 Thus f (n) =O(g(n)). COMP3600/6466 Algorithms 018 Lecture 3 3 O-notation For a given function g(n), we denote by O(g(n)) the set of {f (n) : there exist positive constants c and n 0 such that 0 apple f (n) apple cg(n), for all n n 0 }. If f (n) O(g(n)), we write f (n) =O(g(n)) and call g(n) an asymptotic upper bound for f (n). Intuitively, f (n) =O(g(n)) means that, for large values of n and to within a constant factor, the function g(n) is an upper bound on f (n). In other words, f (n) grows no faster than g(n). O-notation 3 Example. Let f (n) =n 3 and g(n) =.Thenf (n) 6= O(g(n)). To prove this, suppose to the contrary that c > 0 is a constant such that and so 0 apple n 3 apple c, 0 apple n apple c, for all su ciently large values of n. Clearly, regardless of the initial choice of the constant c, theinequalityn apple c cannot hold for arbitrary large n. Thus f (n) 6= O(g(n)). COMP3600/6466 Algorithms 018 Lecture Exercise. Let f (n) =n 3 f (n) =O(g(n)). and g(n) =. Prove or disprove that COMP3600/6466 Algorithms 018 Lecture 4 4

2 -notation For a given function g(n), we denote by (g(n)) the set of {f (n) : there exist positive constants c and n 0 such that 0 apple cg(n) apple f (n), for all n n 0 }. If f (n) (g(n)), we write f (n) = (g(n)) and call g(n) an asymptotic lower bound for f (n). Intuitively, f (n) = (g(n)) means that, for large values of n and to within a constant factor, the function g(n) is a lower bound on f (n). That is, f (n) grows at least as fast as g(n). COMP3600/6466 Algorithms 018 Lecture 5 5 -notation 3 Example: Let f (n) =n ln n + 10n and g(n) =lnn. Then To prove this, we have to find positive constants c and n 0 such that 0 apple c ln n apple n ln n + 10n, 0 apple c apple n + 10n ln n, Since the term on the right-hand side of the last inequality can be made arbitrarily large by choosing large enough n, any choice of c > 0worksforsu ciently large n (e.g., c =1andn 0 =work). Thus Exercise: Let f (n) =n and g(n) =4n 3.Determinewhether COMP3600/6466 Algorithms 018 Lecture 7 7 Example: Let f (n) = 7 n +5n -notation 3andg(n) =n. Then To prove this, we have to find positive constants c and n 0 such that 0 apple cg(n) apple f (n), that is, 0 apple cnapple 7 n +5n 3, 0 apple c apple 7 n +5 3 n, Since 3 n can be made arbitrarily small and 7 n can be made arbitrarily large by choosing large enough n, any choice of c > 0 works for all su ciently large n (e.g., c =1andn 0 =1work). -notation For a given function g(n), we denote by (g(n)) the set of {f (n) : there exist positive constants c 1, c,andn 0 such that 0 apple c 1 g(n) apple f (n) apple c g(n), for all n n 0 }. If f (n) (g(n)), we write f (n) = (g(n)), and we call g(n) an asymptotically tight bound for f (n). Intuitively, f (n) = (g(n)) means that, for large values of n and to within constant factors, the function g(n) bounds f (n) above and below. That is, f (n) has the same rate of growth as g(n). Thus COMP3600/6466 Algorithms 018 Lecture 6 6 COMP3600/6466 Algorithms 018 Lecture 8 8

3 -notation Example: Let f (n) = 1 n 3n +7andg(n) =n.then f (n) = (g(n)). Relationship between O,, and To prove this, we have to determine positive constants c 1, c,and n 0 such that 0 apple c 1 g(n) apple f (n) apple c g(n), that is, 0 apple c 1 n apple 1 n 3n +7apple c n, 0 apple c 1 apple 1 when n n 0. 3 n + 7 n apple c, Cormen et al, p.45 7 Since 3 n can be made arbitrarily small by choosing large n enough values of n, c 1 = 1 4, c = 3 4,andn 0 = 0 make the above inequalities true. Thus f (n) = (g(n)). COMP3600/6466 Algorithms 018 Lecture 9 9 Theorem: For any two f (n) andg(n), we have f (n) = (g(n)) if and only if f (n) =O(g(n)) and Exercise. Prove the theorem. COMP3600/6466 Algorithms 018 Lecture notation 3 Example: Let f (n) = P n i=1 i and g(n) =n.then f (n) = (g(n)). Now f (n) = P n i=1 i = n(n+1). We have to determine positive constants c 1, c,andn 0 such that 0 apple c 1 g(n) apple f (n) apple c g(n), that is, 0 apple c 1 n apple 1 n + 1 n apple c n, 0 apple c 1 apple apple c, Since 1 can be made arbitrarily small by choosing large enough values of n, c 1 = 1, c =1andn 0 = 1 make the above inequalities true. Thus f (n) = (g(n)). COMP3600/6466 Algorithms 018 Lecture Why do we need n 0? Consider the definition of f (n) =O(g(n)). The purpose of the n n 0 condition is to avoid inconvenient behavior for small values of n. One situation is when f (n) is negative for small n. The most important inconvenient behavior in practice is that g(n) might be 0. Example: It is the case that 1 + ln n = O(ln n). But there is no constant c such that 0 apple 1+lnn apple c ln n, whenn = 1. To avoid this inconvenient case when n = 1, we choose n 0 = : 0 apple 1+lnn apple 4lnn, for n. Exercise: Suppose that f (n) =O(g(n)), and f (n) > 0and g(n) > 0, for n 1. Show that there exists a positive constant c such that 0 apple f (n) apple cg(n), for n 1. COMP3600/6466 Algorithms 018 Lecture 1 1

4 Asymptotic notation in equations Exercises f (n) =O(g(n)) has been defined to mean f (n) O(g(n)). Note that here = is not symmetric: f (n) =O(g(n)) does not imply O(g(n)) = f (n) (whatever this might mean). In more general settings, asymptotic notation can be used by applying the following two rules: When asymptotic notation appears only on the right-hand side of an equation (or inequality), it stands for some anonymous function that we do not care to specify. Exercise: Determine the meaning of O(n) =O(n ) and prove that it is valid. Exercise: ShowthatO(n )=O(n) is not valid. Exercise: Prove that n + n O(log n) =O(n log n). Example: 3 +3n +5= 3 + (n ). Interpretation: There is some function f (n) in (n ), namely f (n) =3n + 5, such that 3 +3n +5= 3 + f (n). 13 COMP3600/6466 Algorithms 018 Lecture 13 COMP3600/6466 Algorithms 018 Lecture Asymptotic notation in equations When asymptotic notation appears (also) on the left of an equation, we mean: No matter how the anonymous are chosen on the left of the equal sign, there is a way to choose the anonymous on the right of the equal sign to make the equation valid. Example: 3 + (n )= (n 3 ). Interpretation: For any choice f (n) (n ), there is a function g(n) (n 3 )suchthat 3 + f (n) =g(n). o-notation o-notation is used to denote an upper bound that is not asymptotically tight. For a given function g(n), we denote by o(g(n)) the set of {f (n) : for any positive constant c, there exists a positive constant n 0 such that 0 apple f (n) < cg(n), for all n n 0 }. The condition on the f (n) inthissetisequivalentto f (n) lim n!1 g(n) =0. Intuitively, f (n) = o(g(n)) means that f (n) becomes insignificant relative to g(n), as n approaches infinity. COMP3600/6466 Algorithms 018 Lecture COMP3600/6466 Algorithms 018 Lecture 16 16

5 o-notation Example: If f (n) = and g(n) =n,thenf (n) =o(g(n)). Let c be an arbitrary positive constant. We have to show that that is, 0 apple < cn, 0 apple n < c, for all su ciently large values of n. These inequalities are satisfied whenever n > c,sotheyare satisfied for all su ciently large values of n. Thus f (n) =o(g(n)). Exercise: Prove that 6= o(n ). 3p Exercise: Prove or disprove: 17 5 n = o(n)ċomp3600/6466 Algorithms 018 Lecture 17 Example: If f (n) = n!-notation and g(n) =n, thenf (n) =!(g(n)). Let c be an arbitrary positive constant. We have to prove that 0 apple cn< n, 0 apple c n < 1, for all su ciently large values of n. These inequalities are satisfied whenever n > c, sotheyare satisfied for all su ciently large values of n. Thusf (n) =!(g(n)). Exercise: Prove that n 6=!(n ). Exercise: Prove or disprove: n 3 =!( p n). COMP3600/6466 Algorithms 018 Lecture 19 19!-notation!-notation is used to denote a lower bound that is not asymptotically tight. For a given function g(n), we denote by!(g(n)) the set of {f (n) : for any positive constant c, there exists a positive constant n 0 such that 0 apple cg(n) < f (n), for all n n 0 }. The condition on the f (n) inthissetisequivalentto f (n) lim n!1 g(n) = 1. Intuitively, f (n) =!(g(n)) means that f (n) becomes arbitrarily large relative to g(n), as n approaches infinity. COMP3600/6466 Algorithms 018 Lecture 18 18