Md Momin Al Aziz. Analysis of Algorithms. Asymptotic Notations 3 COMP Computer Science University of Manitoba

Transcription

1 Md Momin Al Aziz Computer Science University of Manitoba Analysis of Algorithms Asymptotic Notations 3 COMP 2080

2 Outline 1. Visualization 2. Little notations 3. Properties of Asymptotic Notations 4. Some popular run times iqmetrix Females in Technology Scholarship Momin Analysis of Algorithms May 24, / 20

3 Visualization Visualize: Asymptotic upper bound Figure: f (n) is O(g(n)) when f (n) cg(n) for constants c, n 0 and n n 0, n 0 0 Source: Introduction to Algorithms (page 45) Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

4 Visualization Visualize: Asymptotic lower bound Figure: f (n) is Ω(g(n)) when f (n) cg(n) for constants c, n 0 and n n 0, n 0 0 Source: Introduction to Algorithms (page 45) Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

5 Visualization Visualize: Asymptotic tight bound Figure: f (n) is Θ(g(n)) when c 1 g(n) f (n) c 2 g(n) for constants c 1, c 2, n 0 and n n 0, n 0 0 Source: Introduction to Algorithms (page 45) Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

6 Visualization Little Ones There are two other notations parallel to Big-O, and Big-Ω which are little-o and ω. These o, ω notations define the weaker upper and lower bound respectively and can be defined as follows: Definition 1 f (n) is o(n) for constant c > 0 and n 0 0 where for all n n 0 and f (n) < cg(n). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

7 Other Notations Little-o Example We have seen that, f (n) = 2n 2 + 3n + 2 is O(n 3 ) Example: f (n) = 2n 2 + 3n + 2 is o(n 3 ) f (n) = 2n 2 + 3n + 2 = 2n 2 + 3n + 2 2n 2 + 3n 2 + 2n 2 ( )n 2 < ( )n 3 [n 0 > 1] < 7n 3 Hence, f (n) = 2n 2 + 3n + 2 is o(n 3 ) Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

8 Other Notations Little-o Example We have seen that, f (n) = 2n 2 + 3n + 2 is O(n 3 ) Example: f (n) = 2n 2 + 3n + 2 is o(n 3 ) f (n) = 2n 2 + 3n + 2 = 2n 2 + 3n + 2 2n 2 + 3n 2 + 2n 2 ( )n 2 < ( )n 3 [n 0 > 1] < 7n 3 Hence, f (n) = 2n 2 + 3n + 2 is o(n 3 ) Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

9 Other Notations Little-o Example We have seen that, f (n) = 2n 2 + 3n + 2 is O(n 3 ) Example: f (n) = 2n 2 + 3n + 2 is o(n 3 ) f (n) = 2n 2 + 3n + 2 = 2n 2 + 3n + 2 2n 2 + 3n 2 + 2n 2 ( )n 2 < ( )n 3 [n 0 > 1] < 7n 3 Hence, f (n) = 2n 2 + 3n + 2 is o(n 3 ) Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

10 Properties of Asymptotic Notations Asymptotic Notations Properties 1. Constant Factor 2. Transitivity 3. Sum 4. Higher orders Momin Analysis of Algorithms May 24, / 20

11 Properties of Asymptotic Notations Constant Factors Constant Factors can be ignored in asymptotic notations: Definition 2 For all k > 0, if kf (n) (or f (kn)) is O(f (n)) Example: f (n) = 4n 2 implies that f (n) is O(n 2 ). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

12 Properties of Asymptotic Notations Constant Factors Constant Factors can be ignored in asymptotic notations: Definition 2 For all k > 0, if kf (n) (or f (kn)) is O(f (n)) Example: f (n) = 4n 2 implies that f (n) is O(n 2 ). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

13 Properties of Asymptotic Notations Transitivity Transitive relation of these asymptotic notations If A > B and B > C, then also A > C If a function f (n) is O(g(n)) and g(n) is O(h(n)), then f (n) is O(h(n)). Similar transitivity holds for all the other notations, Ω, Θ, ω, o. Question: Prove the transitivity of Asymptotic upper bound (Big-O). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

14 Properties of Asymptotic Notations Transitivity Transitive relation of these asymptotic notations If A > B and B > C, then also A > C If a function f (n) is O(g(n)) and g(n) is O(h(n)), then f (n) is O(h(n)). Similar transitivity holds for all the other notations, Ω, Θ, ω, o. Question: Prove the transitivity of Asymptotic upper bound (Big-O). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

15 Properties of Asymptotic Notations Transitivity Transitive relation of these asymptotic notations If A > B and B > C, then also A > C If a function f (n) is O(g(n)) and g(n) is O(h(n)), then f (n) is O(h(n)). Similar transitivity holds for all the other notations, Ω, Θ, ω, o. Question: Prove the transitivity of Asymptotic upper bound (Big-O). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

16 Properties of Asymptotic Notations Transitivity Question: Prove the transitivity of Asymptotic upper bound (Big-O). Proof. From the definition of Big-O, we know that f (n) is O(g(n)) when f (n) c 1 g(n) where some constant c 1 > 0, n n 0. For some different constants c 2, n 0, g(n) = O(h(n)) states that g(n) c 2 h(n). Thus, if we accumulate all two inequality then, f (n) c 1 g(n) c 3 h(n) where c 3 = c 2.c 1 for all n max(n 0, n 0 ). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

17 Properties of Asymptotic Notations Sums of Functions Definition 3 For, f (n) = O(h(n)), g(n) = O(h(n)), f (n) + g(n) is O(h(n)). Proof. For some constants c 1, n 0, asymptotic upper bound states that f (n) c 1 h(n). Similarly, for different constants c 2, n 0, we get g(n) c 2h(n). Now, for an input size, n = max(n 0, n 0 ) we can say f (n) + g(n) (c 1 + c 2 )h(n) = ch(n). Thus, we from definition we can say f (n) + g(n) = O(h(n)). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

18 Properties of Asymptotic Notations Sums of Functions Definition 3 For, f (n) = O(h(n)), g(n) = O(h(n)), f (n) + g(n) is O(h(n)). Proof. For some constants c 1, n 0, asymptotic upper bound states that f (n) c 1 h(n). Similarly, for different constants c 2, n 0, we get g(n) c 2h(n). Now, for an input size, n = max(n 0, n 0 ) we can say f (n) + g(n) (c 1 + c 2 )h(n) = ch(n). Thus, we from definition we can say f (n) + g(n) = O(h(n)). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

19 Properties of Asymptotic Notations Question Question: If g = O(f ), prove that f + g = Θ(f ). Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

20 Properties of Asymptotic Notations Higher Orders The running time of a function will grow faster for higher orders on input size n than the lower ones. Definition 4 f (n) = n r is Θ(n r ) and f (n) is O(n s ) where r < s, and r, s are integers. Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

21 Common Running Times Common Running Times Linear Time (O(n)) Quadratic Time (O(n 2 )) O(n log n) Time Sublinear Time Beyond Polynomial Time Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

22 Common Running Times Linear Time: O(n) Algorithm 1: ArrayMin Input: Array A with n numbers Output: Minimum number (currentmin) from array A 1 currentmin A[0] 2 for i 1 to n 1 do 3 if currentmin > A[i] then 4 currentmin A[i] Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

23 Common Running Times Quadratic Time: O(n 2 ) Algorithm 2: NestedLoop 1 temp 1 2 for i 1 to n do 3 for j 1 to n do 4 temp temp + 1 Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

24 Common Running Times Sub-linear Time and O(n log n) Figure: Linear O(n), O(nlog(n)), and sublinear O(log(n)) time Momin Analysis of Algorithms May 24, / 20

25 Common Running Times Beyond Polynomial Time Two most popular bounds 2 n and n! Travelling Salesman Problem: Given a set of n cities along with their distances, what is the shortest tour that visits all cities? Vehicle Routing Problem: Given a set of n customers along with their distances, what route will be the fastest that covers all n? Momin (azizmma@cs.umanitoba.ca) Analysis of Algorithms May 24, / 20

26 Common Running Times Next class Divide and Conquer Algorithms Momin Analysis of Algorithms May 24, / 20