Copyright (c) 2015-2018 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". Please send corrections (or suggestions) to youngwlim@hotmail.com. This document was produced by using LibreOffice and Octave.
Functions and Ranges x 2 +2 x+1 A 1 = [0,5] x 2 2 x A 2 = [0,100] A 3 = [0,500] 1 3
Small Range A1 All are distinguishable x 2 +2 x+1 x 2 2 x 1 Zoom Out for x > 0.5 x 2 < x 2 +2 x+1 4
Medium Range A2 similar x 2 +2 x+1 x 2 10000+201 10000 2 x Zoom Out More for x > 0.5 x 2 < x 2 +2 x+1 5
Large Range A3 Indistinguishable x 2 +2 x+1 x 2 250000+1001 250000 for x > 0.5 x 2 < x 2 +2 x+1 6
Functions and Ranges 2 x 2 x 2 +2 x+1 2 x B 1 = [0,5] B 2 = [0,100] B 3 = [0,500] 1 7
Small Range, 2x 2 B1 distinguishable 2 x 2 50 x 2 +2 x+1 25+11 2 x 1 Zoom Out for x > 2.414 x 2 +2 x+1 < 2 x 2 for x > k f (n) < C g(n) 8
Medium Range, 2x 2 B2 distinguishable 2 x 2 20000 x 2 +2 x+1 10000+201 2 x 1 Zoom Out for x > 2.414 x 2 +2 x+1 < 2 x 2 for x > k f (n) < C g(n) 9
Large Range, 2x 2 B3 distinguishable 2 x 2 500000 x 2 +2 x+1 250000 +1001 2 x 1 Zoom Out for x > 2.414 x 2 +2 x+1 < 2 x 2 for x > k f (n) < C g(n) 10
Functions and Ranges 10 x 2 x 2 +2 x+1 2 x C 1 = [0,5] C 2 = [0,100] C 3 = [0,500] 1 11
Small Range, 10x 2 C1 distinguishable 10 x 2 250 x 2 +2 x+1 2 x 1 25+11 Zoom Out for x > 0.462 x 2 +2 x+1 < 10 x 2 for x > k f (n) < C g(n) 12
Medium Range, 10x 2 C2 distinguishable 10 x 2 100000 x 2 +2 x+1 10000+201 Zoom Out for x > 0.462 x 2 +2 x+1 < 10 x 2 for x > k f (n) < C g(n) 13
Large Range, 10x 2 C3 distinguishable 10 x 2 2500000 x 2 +2 x+1 250000 +1001 for x > 0.462 x 2 +2 x+1 < 10 x 2 for x > k f (n) < C g(n) 14
Functions and Ranges 10 x x 2 +2 x+1 x 2 D 1 = [0,5] D 2 = [0,100] D 3 = [0,500] 1 15
Small Range, 10x D1 10 x x 2 +2 x+1 distinguishable x 2 1 Zoom Out for 0.127 < x < 7.873 x 2 +2 x+1 < 10 x 16
Medium Range, 10x D2 x 2 +2 x+1 x 2 indistinguishable 10 x Zoom Out for x > 7.873 10 x < x 2 +2 x+1 for x > k C g(n) < f (n) 17
Large Range, 10x D3 x 2 +2 x+1 x 2 indistinguishable 10 x for x > 7.873 10 x < x 2 +2 x+1 for x > k C g(n) < f (n) 18
Big-O Definition Let f and g be functions (Z R or R R) from the set of integers or the set of real numbers to the set of real numbers. We say f(x) is O(g(x)) f(x) is big-oh of g(x) If there are constants C and k such that f(x) C g(x) whenever x > k. g(x) : upper bound of f(x) 19
Big-Ω Definition Let f and g be functions (Z R or R R) from the set of integers or the set of real numbers to the set of real numbers. We say f(x) is Ω(g(x)) f(x) is big-omega of g(x) If there are constants C and k such that C g(x) f(x) whenever x > k. g(x) : lower bound of f(x) 20
Big-O Definition for k < x f (x) C g(x) C g(n) f (n) f (x) is O(g(x)) x=k x g(x) : upper bound of f(x) g(x) has a simpler form than f(x) is usually a single term 21
Big-Ω Definition for k < x f (x) C g(x) f (x) C g(x) f (x) is Ω(g(x)) g(x) : lower bound of f(x) x=k x g(x) has a simpler form than f(x) is usually a single term 22
Big-Θ definition for k < x C g(x) f (x) f (x) C g(x) C 1 g(x) f (x) C 2 g(x) f (x) is O(g(x)) f (x) is Ω(g(x)) f (x) is Θ(g(x)) 23
Big-Θ = Big-Ω Big-O for k < x O(g(x)) C 1 g(x) f (x) C 2 g(x) f (x) is Θ(g(x)) Ω(g(x)) Ω(g(x)) O(g(x)) Θ (g(x)) 24
Θ(x) and Θ(1) for 0 < k < x f (x) C x f (x) f (x) C x C 1 x f (x) C 2 x f (x) is O(x) f (x) is Ω(x) f (x) is Θ (x) C 1 f (x) f (x) C 1 C 1 1 f (x) C 2 1 f (x) is O(1) f (x) is Ω(1) f (x) is Θ (1) f (x) 25
Big-O, Big-Ω, Big-Θ Examples for x > 0.5 1 x 2 < x 2 +2 x+1 lower bound upper bound x 2 +2 x+1 is Ω(x 2 ) x 2 +2 x+1 is Θ( x 2 ) for x > 2.414 x 2 +2 x+1 < 2 x 2 x 2 +2 x+1 is O(x 2 ) upper bound for x > 0.462 x 2 +2 x+1 < 10 x 2 x 2 +2 x+1 is O(x 2 ) for x > 7.873 10 x < x 2 +2 x+1 x 2 +2 x+1 is Ω(x) lower bound 26
Many Larger Upper Bounds x 4 x 2 +2 x+1 is O(x 2 ) x 2 +2 x+1 is O(x 3 ) x 2 +2 x+1 is O(x 4 ) x 3 2 x 2 x 2 +2 x+1 the least upper bound? 27
Many Smaller Lower Bound x 2 +2 x+1 x 2 +2 x+1 is Ω (x 2 ) x 2 +2 x+1 is Ω(x) x 2 +2 x+1 is Ω( x) x 2 +2 x+1 is Ω(log x) x x log x the greatest lower bound? 28
Many Upper and Lower Bounds x 2 +2 x+1 is O(x 2 ) x 2 +2 x+1 C x 2 upper bound the least x 2 +2 x+1 is O(x 3 ) x 2 +2 x+1 C x 3 upper bound x 2 +2 x+1 is O(x 4 ) x 2 +2 x+1 C x 4 upper bound x 2 +2 x+1 is O(x 2 ) x 2 +2 x+1 C x 2 lower bound x 2 +2 x+1 is Ω(x) x 2 +2 x+1 C x x 2 +2 x+1 is Ω( x) x 2 +2 x+1 C x x 2 +2 x+1 is Ω(log x) x 2 +2 x+1 C log x lower bound lower bound lower bound the greatest 29
Simultaneously being lower and upper bound f (x) = x 2 +2 x+1 lower bounds Ω (g( x)) O(g(x)) upper bounds x x log x Θ( g( x)) tight bound x 2 x 2 log x x 3 x 4 x 3 x 30
Big-Θ Examples (1) 2 x 2 x 2 +2 x+1 is O(x 2 ) x 2 +2 x+1 1 x 2 x 2 +2 x+1 is Θ( x 2 ) 0.5 x 2 x 2 +2 x+1 is Ω (x 2 ) Zoom Out 31
Big-Θ Examples (2) 2 x 2 x 2 +2 x+1 1 x 2 x 2 +2 x+1 is Θ( x 2 ) 0.5 x 2 32
Big-Θ Examples (3) 5 x 3 6 x 3 +2 x 2 +4 7 x 3 7 x 3 6 x 3 +2 x 2 +4 6 x 3 5 x 3 indistinguishable 6 x 3 +2 x 2 +4 is Θ( x 3 ) 6 x 3 +2 x 2 +4 6 x 3 33
Tight bound Implications f (x) is Θ(g(x)) f (x) is Θ(g(x)) f (x) is O(g(x)) f (x) is Ω(g(x)) f (x) is Θ(g(x)) f (x) is Θ(g(x)) f (x) is O(g(x)) f (x) is Ω(g(x)) Ω(g( x)) O(g( x)) Θ( g( x)) 34
Common Growth Functions x 2 x log x x x log x 35
Upper bounds x 2 x log x x x log x f 1 ( x) f 1 (x) is O(log x) O( x) O(x) O(x log x) O(x 2 ) 36
Lower bounds f 2 ( x) x 2 x log x x x log x f 2 (x) is Ω (x 2 ) Ω(x log x) Ω(x) Ω( x) Ω(log x) 37
Example 1 f (n)=n 6 +3 n f (n)=2 n +12 f (n)=2 n +3 n f (n)=n n +n f (n)=o(n 6 ) f (n)=o(2 n ) f (n)=o(3 n ) f (n)=o(n n ) f (n)=ω(n) f (n)=ω(1) f (n)=ω(2 n ) f (n)=ω(n) https://discrete.gr/complexity/ 38
Example 2 f (n)=n 6 +3 n f (n)=2 n +12 f (n)=2 n +3 n f (n)=n n +n f (n)=o(n 6 ) f (n)=o(2 n ) f (n)=o(3 n ) f (n)=o(n n ) f (n)=ω(n 6 ) f (n)=ω(2 n ) f (n)=ω(3 n ) f (n)=ω(n n ) f (n)=θ(n 6 ) f (n)=θ(2 n ) f (n)=θ(3 n ) f (n)=θ(n n ) https://discrete.gr/complexity/ 39
Example 3 Θ(n) Θ(n) Θ(n 2 ) Θ(n) O(1) O(n) O(n) O(n 2 ) O(n 3 ) O(1) Θ(1) Θ(1) Θ(n) O(n) O(n 2 ) Θ(n 2 ) O(n 2 ) O(n 3 ) Θ(n) Ω(n) O(1) Θ(1) Ω(1) O(1) log(n) : O(log n) O(n) log(n) : Ω(log n) O(1) https://discrete.gr/complexity/ 40
Example 4 Θ(n) O(n) upper bound tight Θ(n 2 ) Θ(1) Θ(n) O(n 3 ) O(n) O(1) Θ(n) O(2 n) upper bound upper bound upper bound upper bound wrong tight Θ(n) Ω(n) O(1) O(2 n) = O(n) https://discrete.gr/complexity/ 41
Example 5 Θ(1) Θ( n) Θ(n) O(1) O( n) O(n) Θ(n 2 ) O(n 2 ) Θ(n 3 ) O(n 3 ) tight tight tight tight tight upper bound upper bound upper bound upper bound upper bound O(n) O(n) O(n 2 ) O(n 3 ) O(n 4 ) upper bound upper bound upper bound upper bound upper bound Θ(1) Ω(1) tight lower bound Θ( n) Θ(n) Θ(n 2 ) Ω( n) Ω(n) Ω(n 2 ) tight tight tight lower bound lower bound lower bound Ω(1) Ω( n) Ω(n) lower bound lower bound lower bound Θ(n 3 ) Ω(n 3 ) tight lower bound Ω(n 2 ) lower bound https://discrete.gr/complexity/ 42
References [1] http://en.wikipedia.org/ [2]